Multi-metric filtering

ABSTRACT

A filter unit of a video encoder or video decoder can determine a first metric for a group of pixels within a block of pixels based on a comparison of a subset of the pixels in the block to other pixels in the block; determine a filter based on the first metric; and generate a filtered image by applying the filter to the group of pixels. The subset of pixels can be selected to not include pixels on the boundary of the block of pixels.

This application is a continuation of U.S. application Ser. No.13/401,685, filed Feb. 21, 2012 which claims the benefit of U.S.Provisional Application No. 61/445,967, filed Feb. 23, 2011, U.S.Provisional Application No. 61/448,771, filed Mar. 3, 2011, U.S.Provisional Application No. 61/473,713, filed Apr. 8, 2011, U.S.Provisional Application No. 61/476,260, filed Apr. 16, 2011, U.S.Provisional Application No. 61/478,287, filed Apr. 22, 2011, U.S.Provisional Application No. 61/503,426, filed Jun. 30, 2011, U.S.Provisional Application No. 61/503,434, filed Jun. 30, 2011, U.S.Provisional Application No. 61/503,440, filed Jun. 30, 2011, U.S.Provisional Application No. 61/527,463, filed Aug. 25, 2011, and U.S.Provisional Application No. 61/531,571, filed Sep. 6, 2011, the entirecontent of each of which is incorporated herein by reference in theirentirety.

TECHNICAL FIELD

This disclosure relates to block-based digital video coding used tocompress video data and, more particularly to, techniques for thefiltering of video blocks.

BACKGROUND

Digital video capabilities can be incorporated into a wide range ofdevices, including digital televisions, digital direct broadcastsystems, wireless communication devices such as radio telephonehandsets, wireless broadcast systems, personal digital assistants(PDAs), laptop computers, desktop computers, tablet computers, digitalcameras, digital recording devices, video gaming devices, video gameconsoles, and the like. Digital video devices implement videocompression techniques, such as MPEG-2, MPEG-4, or ITU-T H.264/MPEG-4,Part 10, Advanced Video Coding (AVC), to transmit and receive digitalvideo more efficiently. Video compression techniques perform spatial andtemporal prediction to reduce or remove redundancy inherent in videosequences. New video standards, such as the High Efficiency Video Coding(HEVC) standard being developed by the “Joint Collaborative Team-VideoCoding” (JCTVC), which is a collaboration between MPEG and ITU-T,continue to emerge and evolve. This new HEVC standard is also sometimesreferred to as H.265.

Block-based video compression techniques may perform spatial predictionand/or temporal prediction. Intra-coding relies on spatial prediction toreduce or remove spatial redundancy between video blocks within a givenunit of coded video, which may comprise a video frame, a slice of avideo frame, or the like. In contrast, inter-coding relies on temporalprediction to reduce or remove temporal redundancy between video blocksof successive coding units of a video sequence. For intra-coding, avideo encoder performs spatial prediction to compress data based onother data within the same unit of coded video. For inter-coding, thevideo encoder performs motion estimation and motion compensation totrack the movement of corresponding video blocks of two or more adjacentunits of coded video.

A coded video block may be represented by prediction information thatcan be used to create or identify a predictive block, and a residualblock of data indicative of differences between the block being codedand the predictive block. In the case of inter-coding, one or moremotion vectors are used to identify the predictive block of data from aprevious or subsequent coding unit, while in the case of intra-coding,the prediction mode can be used to generate the predictive block basedon data within the CU associated with the video block being coded. Bothintra-coding and inter-coding may define several different predictionmodes, which may define different block sizes and/or predictiontechniques used in the coding. Additional types of syntax elements mayalso be included as part of encoded video data in order to control ordefine the coding techniques or parameters used in the coding process.

After block-based prediction coding, the video encoder may applytransform, quantization and entropy coding processes to further reducethe bit rate associated with communication of a residual block.Transform techniques may comprise discrete cosine transforms (DCTs) orconceptually similar processes, such as wavelet transforms, integertransforms, or other types of transforms. In a discrete cosine transformprocess, as an example, the transform process converts a set of pixeldifference values into transform coefficients, which may represent theenergy of the pixel values in the frequency domain. Quantization isapplied to the transform coefficients, and generally involves a processthat limits the number of bits associated with any given transformcoefficient. Entropy coding comprises one or more processes thatcollectively compress a sequence of quantized transform coefficients.

Filtering of video blocks may be applied as part of the encoding anddecoding loops, or as part of a post-filtering process on reconstructedvideo blocks. Filtering is commonly used, for example, to reduceblockiness or other artifacts common to block-based video coding. Filtercoefficients (sometimes called filter taps) may be defined or selectedin order to promote desirable levels of video block filtering that canreduce blockiness and/or improve the video quality in other ways. A setof filter coefficients, for example, may define how filtering is appliedalong edges of video blocks or other locations within video blocks.Different filter coefficients may cause different levels of filteringwith respect to different pixels of the video blocks. Filtering, forexample, may smooth or sharpen differences in intensity of adjacentpixel values in order to help eliminate unwanted artifacts.

SUMMARY

This disclosure describes techniques associated with filtering of videodata in a video encoding and/or video decoding process. In accordancewith this disclosure, filtering is applied at an encoder, and filterinformation is encoded in the bitstream to enable a decoder to identifythe filtering that was applied at the encoder. The decoder receivesencoded video data that includes the filter information, decodes thevideo data, and applies filtering based on the filtering information. Inthis way, the decoder applies the same filtering that was applied at theencoder. According to the techniques of this disclosure, on aframe-by-frame, slice-by-slice, or LCU-by-LCU basis, an encoder mayselect one or more sets of filters, and on a coded-unit-by-coded-unitbasis, the encoder may determine whether or not to apply filtering. Forthe coded units (CUs) that are to be filtered, the encoder can performfiltering on a pixel-by-pixel or group-by-group basis, where a groupmight, for example, be a 2×2 block of pixels or a 4×4 block of pixels.

In one example, a method of video coding includes determining a firstmetric for a block of pixels, wherein the first metric is determinedbased on a comparison of a subset of the pixels in the block to otherpixels in the block; based on the first metric, determining a filter forthe block of pixels; and, generating a filtered image by applying thefilter to the block of pixels.

In another example, a video coding device includes a filter unitconfigured to determine a first metric for a block of pixels, whereinthe first metric is determined based on a comparison of a subset of thepixels in the block to other pixels in the block, determine a filter forthe block of pixels based on the first metric, and generate a filteredimage by applying the filter to the block of pixels; and a memoryconfigured to store a filtered result of the filter unit;

In another example, a video coding apparatus includes means fordetermining a first metric for a block of pixels, wherein the firstmetric is determined based on a comparison of a subset of the pixels inthe block to other pixels in the block; means for determining a filterfor the block of pixels based on the first metric; and, means forgenerating a filtered image by applying the filter to the block ofpixels.

In another example, a computer-readable storage medium storesinstructions that when executed cause one or more processors todetermine a first metric for a block of pixels, wherein the first metricis determined based on a comparison of a subset of the pixels in theblock to other pixels in the block; determine a filter for the block ofpixels based on the first metric; and, generate a filtered image byapplying the filter to the block of pixels.

The details of one or more examples are set forth in the accompanyingdrawings and the description below. Other features, objects, andadvantages will be apparent from the description and drawings, and fromthe claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an exemplary video encoding anddecoding system.

FIGS. 2A and 2B are conceptual diagrams illustrating an example ofquadtree partitioning applied to a largest coding unit (LCU).

FIGS. 2C and 2D are conceptual diagrams illustrating an example of afilter map for a series of video blocks corresponding to the examplequadtree partitioning of FIGS. 2A and 2B.

FIG. 3 is a block diagram illustrating an exemplary video encoderconsistent with this disclosure.

FIG. 4A is a conceptual diagram illustrating a mapping of ranges for twometrics to filters.

FIG. 4B is a conceptual diagram illustrating a mapping of ranges for anactivity metric and a direction metric to filters.

FIG. 5 is a block diagram illustrating an exemplary video decoderconsistent with this disclosure.

FIGS. 6A, 6B, and 6C show conceptual diagrams of a 4×4 block of pixels.

FIG. 7 is a flow diagram illustrating coding techniques consistent withthis disclosure.

FIGS. 8A and 8B are flow diagrams illustrating coding techniquesconsistent with this disclosure.

FIGS. 9A and 9B are flow diagrams illustrating coding techniquesconsistent with this disclosure.

FIG. 10 is a flow diagram illustrating coding techniques consistent withthis disclosure.

FIG. 11 is a flow diagram illustrating coding techniques consistent withthis disclosure.

DETAILED DESCRIPTION

This disclosure describes techniques associated with filtering of videodata in a video encoding and/or video decoding process. In accordancewith this disclosure, filtering is applied at an encoder, and filterinformation is encoded in the bitstream to enable a decoder to identifythe filtering that was applied at the encoder. The decoder receivesencoded video data that includes the filter information, decodes thevideo data, and applies filtering based on the filtering information. Inthis way, the decoder applies the same filtering that was applied at theencoder. According to the techniques of this disclosure, on aframe-by-frame, slice-by-slice, or LCU-by-LCU basis, an encoder mayselect one or more sets of filters, and on a coded-unit-by-coded-unitbasis, the encoder may determine whether or not to apply filtering. Forthe coded units (CUs) that are to be filtered, the encoder can performfiltering on a pixel-by-pixel or group-by-group basis, where a groupmight, for example, be a 2×2 block of pixels or a 4×4 block of pixels.

According to the techniques of this disclosure, video data can be codedin units referred to as coded units (CUs). CUs can be partitioned intosmaller CUs, or sub-units, using a quadtree partitioning scheme. Syntaxidentifying the quadtree partitioning scheme for a particular CU can betransmitted from an encoder to a decoder. Multiple inputs associatedwith each sub-unit of a given CU can be filtered during the process ofdecoding and reconstructing the encoded video data. According to thetechniques of this disclosure, filter description syntax can describe aset of filters, such as how many filters are in the set or what shapethe filters take. Additional syntax in the bitstream received by thedecoder can identify the filters (i.e. the filter coefficients) used atthe encoder for a particular sub-unit. The filter used for a particularinput can be selected based on two or metrics, where certaincombinations of values for the two or metrics are indexed to specificfilters within a set of filters. In other instances, two or more metricsmay be combined to form a single metric. The mapping of filters tometrics can also be signaled in the bitstream

Different types of filtering may be applied to pixels or blocks ofpixels based on two or more metrics determined for the video data. Thefilter used for a particular pixel can be selected based on two or moremetrics, such as some combination of an activity metric and a directionmetric. An activity metric, for example, may quantify activityassociated with one or more blocks of pixels within the video data. Theactivity metric may comprise a variance metric indicative of pixelvariance within a set of pixels. An activity metric may be eitherdirection-specific or non-direction-specific. For example, anon-direction-specific activity metric may include a sum-modifiedLaplacian value, as explained in greater detail below.

Examples of direction-specific activity metrics include a horizontalactivity metric, a vertical activity metric, a 45-degree activitymetric, and a 135-degree activity metric. A direction metric may for ablock of pixels quantify any of the horizontal activity, verticalactivity, or diagonal activity of a pixel or group of pixels, or adirection metric may include a comparison of horizontal activity,vertical activity, and/or diagonal activity, where horizontal activitygenerally refers to changes in pixel values in a horizontal direction,vertical activity generally refers to changes in pixel values in avertical direction, and diagonal activity generally refers to changes inpixel values in a diagonal direction.

According to techniques of this disclosure, when determining a filterfor a block of pixels, a subset of pixels within the block may be usedto reduce encoding and decoding complexity. For example, whendetermining a filter for a 4×4 block of pixels, it may not be necessaryto use all sixteen pixels of the 4×4 block. Additionally, according totechniques of this disclosure, the subset of pixels from within acurrent block being coded can be selected such that the metrics arecalculated only using pixel values of the current block and not pixelvalues of neighboring blocks. For instance, the metric for a pixel beingevaluated might be calculated based on comparing the pixel to nearbypixels. In some instances, one or more of the nearby pixels for thepixel being evaluated might be in a different block than the pixel beingevaluated. In other instances, however, one of more of the nearby pixelsfor the pixel might be in the same block as the pixel. According totechniques of this disclosure, the subset of pixels can be selected toinclude pixels that do not have nearby pixels in neighboring blocks.Additionally or alternatively, the subset of pixels may include pixelsthat have nearby pixels in neighboring blocks, but those nearby pixelsin neighboring blocks may not be used when determining the metric. Bybasing the determination of a particular metric on pixels within acurrent block and not on pixels of neighboring blocks, the need forbuffers at the encoder and/or decoder may, in some instances, be reducedor even eliminated.

In some instances, according to techniques of this disclosure, thesubset of pixels from within a current block being coded can be selectedsuch that the metrics are calculated only using pixel values of thecurrent block and left and right neighboring blocks but not pixel valuesof upper neighboring blocks or lower neighboring blocks. As a result ofthe raster scan order used when coding video blocks, line buffers forupper and lower neighboring blocks tend to need to store far more pixelvalues than line buffers for storing pixel values of left and rightneighboring blocks.

According to the techniques of this disclosure, a filter unit, such asan adaptive-in loop filter, can be configured to utilize multiplefilters based on multi-metric filter mapping. The multiple filters maybe used in conjunction with a single input or multiple inputs. As willbe described in more detail below, the multiple inputs described in thisdisclosure generally refer to intermediate video block data or imagedata that is produced during the encoding and decoding processes.Multiple inputs associated with a given video block can include, forexample, a reconstructed block or image (RI), a pre-deblockedreconstructed block or image (pRI), a prediction block or image (PI),and/or a quantized prediction error image (EI). In a single inputscheme, a filter may only be applied to one of the inputs above, such asRI. Also, as explained in greater detail below, the filtering techniquesof this disclosure can be applied to CUs of various sizes using aquadtree partitioning scheme. By utilizing multiple filters withmulti-metric filter mapping for CUs partitioned using a quadtreepartitioning scheme, video coding performance, as measured by one orboth of compression rate and reconstructed video quality, might beimproved.

To implement the multi-metric filtering techniques described above, anencoder maintains, by generating, updating, storing, or other means, amapping of combinations of ranges to filters. As one example, thecombination of a first range for a first metric and a first range for asecond metric may map to a first filter. The combination of the firstrange for the first metric and a second range for the second metric mayalso map to the first filter or may map to a second filter. If a firstmetric has eight ranges and a second metric has four ranges, forexample, then the first and second metric can have thirty-twocombinations of ranges, and each of the thirty-two combinations can bemapped to a filter. Each combination, however, is not necessarily mappedto a unique filter. Thus, the thirty-two combinations might map to fourfilters, eight filters, ten filters, or some other number of filters. Inorder to apply the same filters as an encoder, a decoder may alsomaintain the same mappings of range combinations to filters.

This disclosure describes techniques for signaling from an encoder to adecoder, in an encoded bitstream, a mapping of range combinations tofilters. The mapping may, for example, associate each range combinationwith a filter identification (ID). One simple way to signal this mappingis to use one codeword for each filter ID, and then for each combinationof ranges, send the codeword of the corresponding filter ID. Thistechnique, however, is typically inefficient. Techniques of the presentdisclosure may exploit correlations within the mapping by usingdifferential coding methods. Combinations of ranges that share a commonrange sometimes use the same filter. As one example, the combination ofa first range for a first metric and a first rage for a second metricand the combination of the first range for the first metric and a secondrange for the second metric share a common range (the first range of thefirst metric). Thus, these two combinations might, in some instances,map to the same filter ID. By exploiting this correlation, thetechniques of this disclosure may reduce the number of bits needed tosignal the mapping of range combinations to filter IDs from an encoderto a decoder.

In addition to signaling the mapping of range combinations to filterIDs, this disclosure also describes techniques for signaling, in anencoded bitstream, filter coefficients for filters. Techniques of thepresent disclosure include using differential coding methods to signalfilter coefficients from an encoder to a decoder. In this manner, thefilter coefficients for a second filter might be communicated to adecoder as difference information, where the difference informationdescribes how to modify the filter coefficients of a first filter in amanner that produces the filter coefficients of the second filter.Differential coding techniques may be more effective (i.e. may result ina greater savings of bits) when the filter coefficients of the first andsecond filter are more similar than compared to when the filtercoefficients of the first and second filter are less similar. Thetechniques of this disclosure include determining a sequential order inwhich to signal filter coefficients for filters. The orderingsdetermined using the techniques described in this disclosure may resultin improved differential coding of filter coefficients, and thus, may insome instances result in a savings of bits when signaling the filtercoefficients.

Although the techniques of this disclosure may at times be described inreference to in-loop filtering, the techniques may be applied to in-loopfiltering, post-loop filtering, and other filtering schemes such asswitched filtering. In-loop filtering generally refers to filtering inwhich the filtered data is part of the encoding and decoding loops suchthat filtered data is used for predictive intra- or inter-coding.Post-loop filtering refers to filtering that is applied to reconstructedvideo data after the encoding loop. With post-loop filtering, theunfiltered data, as opposed to the filtered data, is used for predictiveintra- or inter-coding. In some implementations, the type of filteringmay switch between post-loop filtering and in-loop filtering on, forexample, a frame-by-frame, slice-by-slice, or other such basis, and thedecision of whether to use post-loop filtering or in-loop filtering canbe signaled from encoder to decoder for each frame, slice, etc. Thetechniques of this disclosure are not limited to in-loop filtering orpost filtering, and may apply to a wide range of filtering appliedduring video coding.

In this disclosure, the term “coding” refers to encoding or decoding.Similarly, the term “coder” generally refers to any video encoder, videodecoder, or combined encoder/decoder (codec). Accordingly, the term“coder” is used herein to refer to a specialized computer device orapparatus that performs video encoding or video decoding.

Additionally, in this disclosure, the term “filter” generally refers toa set of filter coefficients. For example, a 3×3 filter may be definedby a set of 9 filter coefficients, a 5×5 filter may be defined by a setof 25 filter coefficients, a 9×5 filter may be defined by a set of 45filter coefficients, and so on. The term “set of filters” generallyrefers to a group of more than one filter. For example, a set of two 3×3filters, could include a first set of 9 filter coefficients and a secondset of 9 filter coefficients. According to techniques described in thisdisclosure, for a series of video blocks, such as a frame, slice, orlargest coding unit (LCU), information identifying sets of filters aresignaled from the encoder to the decoder in a header for the series ofthe video blocks. The term “shape,” sometimes called the “filtersupport,” generally refers to the number of rows of filter coefficientsand number of columns of filter coefficients for a particular filter.For example, 9×9 is an example of a first shape, 9×5 is an example of asecond shape, and 5×9 is an example of a third shape. In some instances,filters may take non-rectangular shapes including diamond-shapes,diamond-like shapes, circular shapes, circular-like shapes, hexagonalshapes, octagonal shapes, cross shapes, X-shapes, T-shapes, othergeometric shapes, or numerous other shapes or configuration.

FIG. 1 is a block diagram illustrating an exemplary video encoding anddecoding system 110 that may implement techniques of this disclosure. Asshown in FIG. 1, system 110 includes a source device 112 that transmitsencoded video data to a destination device 116 via a communicationchannel 115. Source device 112 and destination device 116 may compriseany of a wide range of devices. In some cases, source device 112 anddestination device 116 may comprise wireless communication devicehandsets, such as so-called cellular or satellite radiotelephones. Thetechniques of this disclosure, however, which apply more generally tofiltering of video data, are not necessarily limited to wirelessapplications or settings, and may be applied to non-wireless devicesincluding video encoding and/or decoding capabilities.

In the example of FIG. 1, source device 112 includes a video source 120,a video encoder 122, a modulator/demodulator (modem) 123 and atransmitter 124. Destination device 116 includes a receiver 126, a modem127, a video decoder 128, and a display device 130. In accordance withthis disclosure, video encoder 122 of source device 112 may beconfigured to select one or more sets of filter coefficients formultiple inputs in a video block filtering process and then encode theselected one or more sets of filter coefficients. Specific filters fromthe one or more sets of filter coefficients may be selected based on oneor more metrics for one or more inputs, and the filter coefficients maybe used to filter the one or more inputs. The filtering techniques ofthis disclosure are generally compatible with any techniques for codingor signaling filter coefficients in an encoded bitstream.

According to the techniques of this disclosure, a device including videoencoder 122 can signal to a device including video decoder 128 one ormore sets of filter coefficients for a series of video blocks, such as aframe or a slice. For the series of video blocks, video encoder 122 may,for example, signal one set of filters to be used with all inputs, ormay signal multiple sets of filters to be used with multiple inputs (oneset per input, for example). Each video block or CU within the series ofvideo blocks can then contain additional syntax to identify which filteror filters of the set of the filters is to be used for each input ofthat video block, or in accordance with the techniques of thisdisclosure, which filter or filters of the set of the filters is to beused can be determined based on two or more metrics associated with oneor more of the inputs.

More specifically, video encoder 122 of source device 112 may select oneor more sets of filters for a series of video blocks, apply filters fromthe set(s) to pixels or groups of pixels of inputs associated with CUsof the series of video blocks during the encoding process, and thenencode the sets of filters (i.e. sets of filter coefficients) forcommunication to video decoder 128 of destination device 116. Videoencoder 122 may determine one or more metrics associated with inputs ofCUs coded in order to select which filter(s) from the set(s) of filtersto use with pixels or groups of pixels for that particular CU. Videoencoder 122 may also signal to video decoder 128, as part of the codedbitstream, a mapping of combinations of ranges to filters within a setof filters.

On the decoder side, video decoder 128 may determine the filtercoefficients based on filter information received in the bitstreamsyntax. Video decoder 128 may decode the filter coefficients based ondirect decoding or predictive decoding depending upon how the filtercoefficients were encoded, which may be signaled as part of thebitstream syntax. Additionally, the bitstream may include filterdescription syntax information to describe the filters for a set offilters. Based on the filter description syntax, decoder 128 canreconstruct the filter coefficients based on additional informationreceived from encoder 122. The illustrated system 110 of FIG. 1 ismerely exemplary. The filtering techniques of this disclosure may beperformed by any encoding or decoding devices. Source device 112 anddestination device 116 are merely examples of coding devices that cansupport such techniques. Video decoder 128 may also determine themapping of combinations of ranges to filters based on filter informationreceived in the bitstream syntax.

Video encoder 122 of source device 112 may encode video data receivedfrom video source 120 using the techniques of this disclosure. Videosource 120 may comprise a video capture device, such as a video camera,a video archive containing previously captured video, or a video feedfrom a video content provider. As a further alternative, video source120 may generate computer graphics-based data as the source video, or acombination of live video, archived video, and computer-generated video.In some cases, if video source 120 is a video camera, source device 112and destination device 116 may form so-called camera phones or videophones. In each case, the captured, pre-captured or computer-generatedvideo may be encoded by video encoder 122.

Once the video data is encoded by video encoder 122, the encoded videoinformation may then be modulated by modem 123 according to acommunication standard, e.g., such as code division multiple access(CDMA), frequency division multiple access (FDMA), orthogonal frequencydivision multiplexing (OFDM), or any other communication standard ortechnique, and transmitted to destination device 116 via transmitter124. Modem 123 may include various mixers, filters, amplifiers or othercomponents designed for signal modulation. Transmitter 124 may includecircuits designed for transmitting data, including amplifiers, filters,and one or more antennas.

Receiver 126 of destination device 116 receives information over channel115, and modem 127 demodulates the information. The video decodingprocess performed by video decoder 128 may include filtering, e.g., aspart of the in-loop decoding or as a post filtering step following thedecoding loop. Either way, the set of filters applied by video decoder128 for a particular slice or frame may be decoded using the techniquesof this disclosure. Decoded filter information may include identifyingfilter description syntax in the coded bitstream. If, for example,predictive coding is used for the filter coefficients, similaritiesbetween different filter coefficients may be exploited to reduce theamount of information conveyed over channel 115. In particular, a filter(i.e. a set of the filter coefficients) can be predictively coded asdifference values relative to another set of the filter coefficientsassociated with a different filter. The different filter may, forexample, be associated with a different slice or frame. In such a case,video decoder 128 might receive an encoded bitstream comprising videoblocks and filter information that identifies the different frame orslice with which the different filter is associated filter. The filterinformation also includes difference values that define the currentfilter relative to the filter of the different CU. In particular, thedifference values may comprise filter coefficient difference values thatdefine filter coefficients for the current filter relative to filtercoefficients of a different filter used for a different CU.

Video decoder 128 decodes the video blocks, generates the filtercoefficients, and filters the decoded video blocks based on thegenerated filter coefficients. Video decoder 128 can generate the filtercoefficients based on filter description syntax retrieved from thebitstream. The decoded and filtered video blocks can be assembled intovideo frames to form decoded video data. Display device 128 displays thedecoded video data to a user, and may comprise any of a variety ofdisplay devices such as a cathode ray tube (CRT), a liquid crystaldisplay (LCD), a plasma display, an organic light emitting diode (OLED)display, or another type of display device.

Communication channel 115 may comprise any wireless or wiredcommunication medium, such as a radio frequency (RF) spectrum or one ormore physical transmission lines, or any combination of wireless andwired media. Communication channel 115 may form part of a packet-basednetwork, such as a local area network, a wide-area network, or a globalnetwork such as the Internet. Communication channel 115 generallyrepresents any suitable communication medium, or collection of differentcommunication media, for transmitting video data from source device 112to destination device 116. Again, FIG. 1 is merely exemplary and thetechniques of this disclosure may apply to video coding settings (e.g.,video encoding or video decoding) that do not necessarily include anydata communication between the encoding and decoding devices. In otherexamples, data could be retrieved from a local memory, streamed over anetwork, or the like.

Alternatively, encoded data may be output from video encoder 122 to astorage device 132. Similarly, encoded data may be accessed from storagedevice 132 by video decoder 128. Storage device 132 may include any of avariety of distributed or locally accessed data storage media such as ahard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile ornon-volatile memory, or any other suitable digital storage media forstoring encoded video data. In a further example, storage device 132 maycorrespond to a file server or another intermediate storage device thatmay hold the encoded video generated by source device 112. Destinationdevice 116 may access stored video data from storage device 132 viastreaming or download. The file server may be any type of server capableof storing encoded video data and transmitting that encoded video datato the destination device 116. Example file servers include a web server(e.g., for a website), an FTP server, network attached storage (NAS)devices, or a local disk drive. Destination device 14 may access theencoded video data through any standard data connection, including anInternet connection. This may include a wireless channel (e.g., a Wi-Ficonnection), a wired connection (e.g., DSL, cable modem, etc.), or acombination of both that is suitable for accessing encoded video datastored on a file server. The transmission of encoded video data fromstorage device 132 may be a streaming transmission, a downloadtransmission, or a combination of both.

The techniques of this disclosure are not necessarily limited towireless applications or settings. The techniques may be applied tovideo coding in support of any of a variety of multimedia applications,such as over-the-air television broadcasts, cable televisiontransmissions, satellite television transmissions, streaming videotransmissions, e.g., via the Internet, encoding of digital video forstorage on a data storage medium, decoding of digital video stored on adata storage medium, or other applications. In some examples, system 110may be configured to support one-way or two-way video transmission tosupport applications such as video streaming, video playback, videobroadcasting, and/or video telephony.

Video encoder 122 and video decoder 128 may operate according to a videocompression standard such as the ITU-T H.264 standard, alternativelyreferred to as MPEG-4, Part 10, Advanced Video Coding (AVC), which willbe used in parts of this disclosure for purposes of explanation.However, many of the techniques of this disclosure may be readilyapplied to any of a variety of other video coding standards, includingthe newly emerging HEVC standard. Generally, any standard that allowsfor filtering at the encoder and decoder may benefit from variousaspects of the teaching of this disclosure.

Although not shown in FIG. 1, in some aspects, video encoder 122 andvideo decoder 128 may each be integrated with an audio encoder anddecoder, and may include appropriate MUX-DEMUX units, or other hardwareand software, to handle encoding of both audio and video in a commondata stream or separate data streams. If applicable, MUX-DEMUX units mayconform to the ITU H.223 multiplexer protocol, or other protocols suchas the user datagram protocol (UDP).

Video encoder 122 and video decoder 128 each may be implemented as oneor more microprocessors, digital signal processors (DSPs), applicationspecific integrated circuits (ASICs), field programmable gate arrays(FPGAs), discrete logic, software, hardware, firmware or anycombinations thereof. Each of video encoder 122 and video decoder 128may be included in one or more encoders or decoders, either of which maybe integrated as part of a combined encoder/decoder (CODEC) in arespective mobile device, subscriber device, broadcast device, server,or the like.

In some cases, devices 112, 116 may operate in a substantiallysymmetrical manner. For example, each of devices 112, 116 may includevideo encoding and decoding components. Hence, system 110 may supportone-way or two-way video transmission between video devices 112, 116,e.g., for video streaming, video playback, video broadcasting, or videotelephony.

During the encoding process, video encoder 122 may execute a number ofcoding techniques or steps. In general, video encoder 122 operates onvideo blocks within individual video frames in order to encode the videodata. In one example, a video block may correspond to a macroblock or apartition of a macroblock. Macroblocks are one type of video blockdefined by the ITU H.264 standard and other standards. Macroblockstypically refer to 16×16 blocks of data, although the term is alsosometimes used generically to refer to any video block of N×N or N×Msize. The ITU-T H.264 standard supports intra prediction in variousblock sizes, such as 16×16, 8×8, or 4×4 for luma components, and 8×8 forchroma components, as well as inter prediction in various block sizes,such as 16×16, 16×8, 8×16, 8×8, 8×4, 4×8 and 4×4 for luma components andcorresponding scaled sizes for chroma components. In this disclosure,“N×N” refers to the pixel dimensions of the block in terms of verticaland horizontal dimensions, e.g., 16×16 pixels. In general, a 16×16 blockwill have 16 pixels in a vertical direction and 16 pixels in ahorizontal direction. Likewise, an N×N block generally has N pixels in avertical direction and N pixels in a horizontal direction, where Nrepresents a positive integer value. The pixels in a block may bearranged in rows and columns.

The emerging HEVC standard defines new terms for video blocks. Inparticular, video blocks (or partitions thereof) may be referred to as“coding units” (or CUs). With the HEVC standard, largest coded units(LCUs) may be divided into smaller CUs according to a quadtreepartitioning scheme, and the different CUs that are defined in thescheme may be further partitioned into so-called prediction units (PUs).The LCUs, CUs, and PUs are all video blocks within the meaning of thisdisclosure. Other types of video blocks may also be used, consistentwith the HEVC standard or other video coding standards. Thus, the phrase“video blocks” refers to any size of video block. Separate CUs may beincluded for luma components and scaled sizes for chroma components fora given pixel, although other color spaces could also be used.

Video blocks may have fixed or varying sizes, and may differ in sizeaccording to a specified coding standard. Each video frame may include aplurality of slices. Each slice may include a plurality of video blocks,which may be arranged into partitions, also referred to as sub-blocks.In accordance with the quadtree partitioning scheme referenced above anddescribed in more detail below, an N/2×N/2 first CU may comprise asub-block of an N×N LCU, an N/4×N/4 second CU may also comprise asub-block of the first CU. An N/8×N/8 PU may comprise a sub-block of thesecond CU. Similarly, as a further example, block sizes that are lessthan 16×16 may be referred to as partitions of a 16×16 video block or assub-blocks of the 16×16 video block. Likewise, for an N×N block, blocksizes less than N×N may be referred to as partitions or sub-blocks ofthe N×N block. Video blocks may comprise blocks of pixel data in thepixel domain, or blocks of transform coefficients in the transformdomain, e.g., following application of a transform such as a discretecosine transform (DCT), an integer transform, a wavelet transform, or aconceptually similar transform to the residual video block datarepresenting pixel differences between coded video blocks and predictivevideo blocks. In some cases, a video block may comprise blocks ofquantized transform coefficients in the transform domain.

Syntax data within a bitstream may define an LCU for a frame or a slice,which is a largest coding unit in terms of the number of pixels for thatframe or slice. In general, an LCU or CU has a similar purpose to amacroblock coded according to H.264, except that LCUs and CUs do nothave a specific size distinction. Instead, an LCU size can be defined ona frame-by-frame or slice-by-slice basis, and an LCU be split into CUs.In general, references in this disclosure to a CU may refer to an LCU ofa picture or a sub-CU of an LCU. An LCU may be split into sub-CUs, andeach sub-CU may be split into sub-CUs. Syntax data for a bitstream maydefine a maximum number of times an LCU may be split, referred to as CUdepth. Accordingly, a bitstream may also define a smallest coding unit(SCU). This disclosure also uses the terms “block” and “video block” torefer to any of an LCU, CU, PU, SCU, or TU.

As introduced above, an LCU may be associated with a quadtree datastructure. In general, a quadtree data structure includes one node perCU, where a root node corresponds to the LCU. If a CU is split into foursub-CUs, the node corresponding to the CU includes four leaf nodes, eachof which corresponds to one of the sub-CUs. Each node of the quadtreedata structure may provide syntax data for the corresponding CU. Forexample, a node in the quadtree may include a split flag, indicatingwhether the CU corresponding to the node is split into sub-CUs. Syntaxelements for a CU may be defined recursively, and may depend on whetherthe CU is split into sub-CUs.

A CU that is not split may include one or more prediction units (PUs).In general, a PU represents all or a portion of the corresponding CU,and includes data for retrieving a reference sample for the PU. Forexample, when the PU is intra-mode encoded, the PU may include datadescribing an intra-prediction mode for the PU. As another example, whenthe PU is inter-mode encoded, the PU may include data defining a motionvector for the PU. The data defining the motion vector may describe, forexample, a horizontal component of the motion vector, a verticalcomponent of the motion vector, a resolution for the motion vector(e.g., one-quarter pixel precision or one-eighth pixel precision), areference frame to which the motion vector points, and/or a referencelist (e.g., list 0 or list 1) for the motion vector. Data for the CUdefining the PU(s) may also describe, for example, partitioning of theCU into one or more PUs. Partitioning modes may differ between whetherthe CU is uncoded, intra-prediction mode encoded, or inter-predictionmode encoded.

A CU having one or more PUs may also include one or more transform units(TUs). The TUs comprise the data structure that includes residualtransform coefficients, which are typically quantized. In particular,following prediction using a PU, a video encoder may calculate residualvalues for the portion of the CU corresponding to the PU. The residualvalues may be transformed, quantized, scanned and stored in a TU, whichmay have variable sizes corresponding to the size of the transform thatwas performed. Accordingly, a TU is not necessarily limited to the sizeof a PU. Thus, TUs may be larger or smaller than corresponding PUs forthe same CU. In some examples, the maximum size of a TU may be the sizeof the corresponding CU. Again, the TUs may comprise the data structuresthat include the residual transform coefficients associated with a givenCU.

FIGS. 2A and 2B are conceptual diagrams illustrating an example quadtree250 and a corresponding LCU 272. FIG. 2A depicts an example quadtree250, which includes nodes arranged in a hierarchical fashion. Each nodein a quadtree, such as quadtree 250, may be a leaf node with nochildren, or have four child nodes. In the example of FIG. 2A, quadtree250 includes root node 252. Root node 252 has four child nodes,including leaf nodes 256A-256C (leaf nodes 256) and node 254. Becausenode 254 is not a leaf node, node 254 includes four child nodes, whichin this example, are leaf nodes 258A-258D (leaf nodes 258).

Quadtree 250 may include data describing characteristics of acorresponding LCU, such as LCU 272 in this example. For example,quadtree 250, by its structure, may describe splitting of the LCU intosub-CUs. Assume that LCU 272 has a size of 2N×2N. LCU 272, in thisexample, has four sub-CUs 276A-276C (sub-CUs 276) and 274, each of sizeN×N. Sub-CU 274 is further split into four sub-CUs 278A-278D (sub-CUs278), each of size N/2×N/2. The structure of quadtree 250 corresponds tothe splitting of LCU 272, in this example. That is, root node 252corresponds to LCU 272, leaf nodes 256 correspond to sub-CUs 276, node254 corresponds to sub-CU 274, and leaf nodes 258 correspond to sub-CUs278.

Data for nodes of quadtree 250 may describe whether the CU correspondingto the node is split. If the CU is split, four additional nodes may bepresent in quadtree 250. In some examples, a node of a quadtree may beimplemented similar to the following pseudocode:

quadtree_node {    boolean split_flag(1);    // signaling data    if(split_flag) {       quadtree_node child1;       quadtree_node child2;      quadtree_node child3;       quadtree_node child4;    } }The split_flag value may be a one-bit value representative of whetherthe CU corresponding to the current node is split. If the CU is notsplit, the split_flag value may be ‘0’, while if the CU is split, thesplit_flag value may be ‘1’. With respect to the example of quadtree250, an array of split_flag values may be 101000000.

In some examples, each of sub-CUs 276 and sub-CUs 278 may beintra-prediction encoded using the same intra-prediction mode.Accordingly, video encoder 122 may provide an indication of theintra-prediction mode in root node 252. Moreover, certain sizes ofsub-CUs may have multiple possible transforms for a particularintra-prediction mode. Video encoder 122 may provide an indication ofthe transform to use for such sub-CUs in root node 252. For example,sub-CUs of size N/2×N/2 may have multiple possible transforms available.Video encoder 122 may signal the transform to use in root node 252.Accordingly, video decoder 128 may determine the transform to apply tosub-CUs 278 based on the intra-prediction mode signaled in root node 252and the transform signaled in root node 252.

As such, video encoder 122 need not signal transforms to apply tosub-CUs 276 and sub-CUs 278 in leaf nodes 256 and leaf nodes 258, butmay instead simply signal an intra-prediction mode and, in someexamples, a transform to apply to certain sizes of sub-CUs, in root node252, in accordance with the techniques of this disclosure. In thismanner, these techniques may reduce the overhead cost of signalingtransform functions for each sub-CU of an LCU, such as LCU 272.

In some examples, intra-prediction modes for sub-CUs 276 and/or sub-CUs278 may be different than intra-prediction modes for LCU 272. Videoencoder 122 and video decoder 130 may be configured with functions thatmap an intra-prediction mode signaled at root node 252 to an availableintra-prediction mode for sub-CUs 276 and/or sub-CUs 278. The functionmay provide a many-to-one mapping of intra-prediction modes availablefor LCU 272 to intra-prediction modes for sub-CUs 276 and/or sub-CUs278.

A slice may be divided into video blocks (or LCUs) and each video blockmay be partitioned according to the quadtree structure described inrelation to FIGS. 2A-B. Additionally, as shown in FIG. 2C, the quadtreesub-blocks indicated by “ON” may be filtered by loop filters describedherein, while quadtree sub-blocks indicated by “OFF” may not befiltered. The decision of whether or not to filter a given block orsub-block may be determined at the encoder by comparing the filteredresult and the non-filtered result relative to the original block beingcoded. FIG. 2D is a decision tree representing partitioning decisionsthat results in the quadtree partitioning shown in FIG. 2C. The actualfiltering applied to any pixels for “ON” blocks, may be determined basedon the metrics discussed herein.

In particular, FIG. 2C may represent a relatively large video block thatis partitioned according to a quadtree portioning scheme into smallervideo blocks of varying sizes. Each video block is labelled (on or off)in FIG. 2C, to illustrate whether filtering should be applied or avoidedfor that video block. The video encoder may define this filter map bycomparing filtered and unfiltered versions of each video block to theoriginal video block being coded.

Again, FIG. 2D is a decision tree corresponding to partitioningdecisions that result in the quadtree partitioning shown in FIG. 2C. InFIG. 2D, each circle may correspond to a CU. If the circle includes a“1” flag, then that CU is further partitioned into four more CUs, but ifthe circle includes a “0” flag, then that CU is not partitioned anyfurther. Each circle (e.g., corresponding to CUs) also includes anassociated diamond. If the flag in the diamond for a given CU is set to1, then filtering is turned “ON” for that CU, but if the flag in thediamond for a given CU is set to 0, then filtering is turned off. Inthis manner, FIGS. 2C and 2D may be individually or collectively viewedas a filter map that can be generated at an encoder and communicated toa decoder at least once per slice of encoded video data in order tocommunicate the level of quadtree partitioning for a given video block(e.g., an LCU) whether or not to apply filtering to each partitionedvideo block (e.g., each CU within the LCU).

Smaller video blocks can provide better resolution, and may be used forlocations of a video frame that include high levels of detail. Largervideo blocks can provide greater coding efficiency, and may be used forlocations of a video frame that include a low level of detail. A slicemay be considered to be a plurality of video blocks and/or sub-blocks.Each slice may be an independently decodable series of video blocks of avideo frame. Alternatively, frames themselves may be decodable series ofvideo blocks, or other portions of a frame may be defined as decodableseries of video blocks. The term “series of video blocks” may refer toany independently decodable portion of a video frame such as an entireframe, a slice of a frame, a group of pictures (GOP) also referred to asa sequence, or another independently decodable unit defined according toapplicable coding techniques. Aspects of this disclosure might bedescribed in reference to frames or slices, but such references aremerely exemplary. It should be understood that generally any series ofvideo blocks may be used instead of a frame or a slice.

Syntax data may be defined on a per-coded-unit basis such that each CUincludes associated syntax data. The filter information described hereinmay be part of such syntax for a CU, but might more likely be part ofsyntax for a series of video blocks, such as a frame, a slice, a GOP,LCU, or a sequence of video frames, instead of for a CU. The syntax datacan indicate the set or sets of filters to be used with CUs of the sliceor frame. Additionally, not all filter information necessarily has to beincluded in the header of a common series of video blocks. For example,filter description syntax might be transmitted in a frame header, whileother filter information is signaled in a header for an LCU.

Video encoder 122 may perform predictive coding in which a video blockbeing coded is compared to a predictive frame (or other CU) in order toidentify a predictive block. The differences between the current videoblock being coded and the predictive block are coded as a residualblock, and prediction syntax is used to identify the predictive block.The residual block may be transformed and quantized. Transformtechniques may comprise a DCT process or conceptually similar process,integer transforms, wavelet transforms, or other types of transforms. Ina DCT process, as an example, the transform process converts a set ofpixel values into transform coefficients, which may represent the energyof the pixel values in the frequency domain. Quantization is typicallyapplied to the transform coefficients, and generally involves a processthat limits the number of bits associated with any given transformcoefficient.

Following transform and quantization, entropy coding may be performed onthe quantized and transformed residual video blocks. Syntax elements,such as the filter information and prediction vectors defined during theencoding, may also be included in the entropy coded bitstream for eachCU. In general, entropy coding comprises one or more processes thatcollectively compress a sequence of quantized transform coefficientsand/or other syntax information. Scanning techniques, such as zig-zagscanning techniques, are performed on the quantized transformcoefficients, e.g., as part of the entropy coding process, in order todefine one or more serialized one-dimensional vectors of coefficientsfrom two-dimensional video blocks. Other scanning techniques, includingother scan orders or adaptive scans, may also be used, and possiblysignaled in the encoded bitstream. In any case, the scanned coefficientsare then entropy coded along with any syntax information, e.g., viacontent adaptive variable length coding (CAVLC), context adaptive binaryarithmetic coding (CABAC), or another entropy coding process.

As part of the encoding process, encoded video blocks may be decoded inorder to generate the video data used for subsequent prediction-basedcoding of subsequent video blocks. At this stage, filtering may beperformed in order to improve video quality, and e.g., remove blockinessartifacts from decoded video. The filtered data may be used forprediction of other video blocks, in which case the filtering isreferred to as “in-loop” filtering. Alternatively, prediction of othervideo blocks may be based on unfiltered data, in which case thefiltering is referred to as “post filtering.”

On a frame-by-frame, slice-by-slice, or LCU-by-LCU basis, video encoder122 may select one or more sets of filters, and on acoded-unit-by-coded-unit basis, the encoder may determine whether or notto apply filtering. For the CUs that are to be filtered, the encoder canperform filtering on a pixel-by-pixel or group-by-group basis, where agroup might, for example, be a 2×2 block of pixels or a 4×4 block ofpixels. These selections can be made in a manner that promotes the videoquality. Such sets of filters may be selected from pre-defined sets offilters, or may be adaptively defined to promote video quality. As anexample, video encoder 122 may select or define several sets of filtersfor a given frame or slice such that different filters are used fordifferent pixels or groups of pixels of CUs of that frame or slice. Inparticular, for each input associated with a CU, several sets of filtercoefficients may be defined, and the two or more metrics associated withthe pixels of the CU may be used to determine which filter from the setof filters to use with such pixels or groups of pixels.

In some cases, video encoder 122 may apply several sets of filtercoefficients and select one or more sets that produce the best qualityvideo in terms of amount of distortion between a coded block and anoriginal block, and/or the highest levels of compression. In any case,once selected, the set of filter coefficients applied by video encoder122 for each CU may be encoded and communicated to video decoder 128 ofdestination device 118 so that video decoder 128 can apply the samefiltering that was applied during the encoding process for each givenCU.

When two or more metrics are used for determining which filter to usewith a particular input for a CU, the selection of the filter for thatparticular CU does not necessarily need to be communicated to videodecoder 128. Instead, video decoder 128 can also calculate the two ormore metrics, and based on filter information previously provided byvideo encoder 122, match the combination of two or more metrics to aparticular filter.

FIG. 3 is a block diagram illustrating a video encoder 350 consistentwith this disclosure. Video encoder 350 may correspond to video encoder122 of device 120, or a video encoder of a different device. As shown inFIG. 3, video encoder 350 includes a prediction module 332, adders 348and 351, and a memory 334. Video encoder 350 also includes a transformunit 338 and a quantization unit 340, as well as an inverse quantizationunit 342 and an inverse transform unit 344. Video encoder 350 alsoincludes a deblocking filter 347 and an adaptive filter unit 349. Videoencoder 350 also includes an entropy encoding unit 346. Filter unit 349of video encoder 350 may perform filtering operations and also mayinclude a filter selection unit (FSU) 353 for identifying a desirable orpreferred filter or set of filters to be used for decoding. Filter unit349 may also generate filter information identifying the selectedfilters so that the selected filters can be efficiently communicated asfilter information to another device to be used during a decodingoperation.

During the encoding process, video encoder 350 receives a video block,such as an LCU, to be coded, and prediction module 332 performspredictive coding techniques on the video block. Using the quadtreepartitioning scheme discussed above, prediction module 332 can partitionthe video block and perform predictive coding techniques on CUs ofdifferent sizes. For inter coding, prediction module 332 compares thevideo block to be encoded, including sub-blocks of the video block, tovarious blocks in one or more video reference frames or slices in orderto define a predictive block. For intra coding, prediction module 332generates a predictive block based on neighboring data within the sameCU. Prediction module 332 outputs the prediction block and adder 348subtracts the prediction block from the video block being coded in orderto generate a residual block.

For inter coding, prediction module 332 may comprise motion estimationand motion compensation units that identify a motion vector that pointsto a prediction block and generates the prediction block based on themotion vector. Typically, motion estimation is considered the process ofgenerating the motion vector, which estimates motion. For example, themotion vector may indicate the displacement of a predictive block withina predictive frame relative to the current block being coded within thecurrent frame. Motion compensation is typically considered the processof fetching or generating the predictive block based on the motionvector determined by motion estimation. For intra coding, predictionmodule 332 generates a predictive block based on neighboring data withinthe same CU. One or more intra-prediction modes may define how an intraprediction block can be defined.

After prediction module 332 outputs the prediction block and adder 348subtracts the prediction block from the video block being coded in orderto generate a residual block, transform unit 338 applies a transform tothe residual block. The transform may comprise a discrete cosinetransform (DCT) or a conceptually similar transform such as that definedby a coding standard such as the HEVC standard. Wavelet transforms,integer transforms, sub-band transforms or other types of transformscould also be used. In any case, transform unit 338 applies thetransform to the residual block, producing a block of residual transformcoefficients. The transform may convert the residual information from apixel domain to a frequency domain.

Quantization unit 340 then quantizes the residual transform coefficientsto further reduce bit rate. Quantization unit 340, for example, maylimit the number of bits used to code each of the coefficients. Afterquantization, entropy encoding unit 346 scans the quantized coefficientblock from a two-dimensional representation to one or more serializedone-dimensional vectors. The scan order may be pre-programmed to occurin a defined order (such as zig-zag scanning, horizontal scanning,vertical scanning, combinations, or another pre-defined order), orpossibly adaptive defined based on previous coding statistics.

Following this scanning process, entropy encoding unit 346 encodes thequantized transform coefficients (along with any syntax elements)according to an entropy coding methodology, such as CAVLC or CABAC, tofurther compress the data. Syntax elements included in the entropy codedbitstream may include prediction syntax from prediction module 332, suchas motion vectors for inter coding or prediction modes for intra coding.Syntax elements included in the entropy coded bitstream may also includefilter information from filter unit 349, which can be encoded in themanner described herein.

CAVLC is one type of entropy encoding technique supported by the ITUH.264/MPEG4, AVC standard, which may be applied on a vectorized basis byentropy encoding unit 346. CAVLC uses variable length coding (VLC)tables in a manner that effectively compresses serialized “runs” oftransform coefficients and/or syntax elements. CABAC is another type ofentropy coding technique supported by the ITU H.264/MPEG4, AVC standard,which may be applied on a vectorized basis by entropy encoding unit 346.CABAC involves several stages, including binarization, context modelselection, and binary arithmetic coding. In this case, entropy encodingunit 346 codes transform coefficients and syntax elements according toCABAC. Like the ITU H.264/MPEG4, AVC standard, the emerging HEVCstandard may also support both CAVLC and CABAC entropy coding.Furthermore, many other types of entropy coding techniques also exist,and new entropy coding techniques will likely emerge in the future. Thisdisclosure is not limited to any specific entropy coding technique.

Following the entropy coding by entropy encoding unit 346, the encodedvideo may be transmitted to another device or archived for latertransmission or retrieval. Again, the encoded video may comprise theentropy coded vectors and various syntax, which can be used by thedecoder to properly configure the decoding process. Inverse quantizationunit 342 and inverse transform unit 344 apply inverse quantization andinverse transform, respectively, to reconstruct the residual block inthe pixel domain. Summer 351 adds the reconstructed residual block tothe prediction block produced by prediction module 332 to produce apre-deblocked reconstructed video block, sometimes referred to aspre-deblocked reconstructed image. De-blocking filter 347 may applyfiltering to the pre-deblocked reconstructed video block to improvevideo quality by removing blockiness or other artifacts. The output ofthe de-blocking filter 347 can be referred to as a post-deblocked videoblock, reconstructed video block, or reconstructed image.

Filter unit 349 can be configured to receive a single input or multipleinputs. In the example of FIG. 3, filter unit 349 receives as input thepost-deblocked reconstructed image (RI), pre-deblocked reconstructedimage (pRI), the prediction image (PI), and the reconstructed residualblock (EI). Filter unit 349 can use any of these inputs eitherindividually or in combination to produce a reconstructed image to storein memory 334. Additionally, as will be discussed in more detail below,based on two or more metrics, one or more filters can be selected to beapplied to the input(s). In one example, the output of filter unit 349may be one additional filter applied to RI. In another example, theoutput of filter unit 349 may be one additional filter applied to pRI.In other examples, however, the output of filter unit 349 may be basedon multiple inputs. For example, filter unit 349 may apply a firstfilter to pRI and then use the filtered version of pRI in conjunctionwith filtered versions of EI and PI to create a reconstructed image. Ininstances where the output of filter unit 349 is the product of oneadditional filter being applied to a single input, filter unit 349 mayin fact apply filters to the other inputs, but those filters might haveall zero coefficients. Similarly, if the output of filter unit 349 isthe product of applying three filters to three inputs, filter unit 349may in fact apply a filter to the fourth input, but that filter mighthave all zero coefficients.

Filter unit 349 may also be configured to receive a single input. Forexample, although FIG. 3 shows PI, EI, pRI, and RI being input intofilter unit 349, in some implementations RI might be the only inputreceived by filter unit 349. In such an implementation, filter unit 349might apply a filter to RI so that a filtered version of RI is moresimilar to the original image than the unfiltered version of RI. Inother implementations, filter unit 349 and de-blocking filter 347 may becombined into a single filtering unit that applies filtering to pRI. Thetechniques of this disclosure, which generally relate tomulti-metric-based filter mapping, are compatible with both single-inputand multi-input filtering schemes that utilize multiple filters.

Filtering by filter unit 349 may improve compression by generatingpredictive video blocks that more closely match video blocks being codedthan unfiltered predictive video blocks. After filtering, thereconstructed video block may be used by prediction module 332 as areference block to inter-code a block in a subsequent video frame orother CU. Although filter unit 349 is shown “in-loop,” the techniques ofthis disclosure could also be used with post filters, in which casenon-filtered data (rather than filtered data) would be used for purposesof predicting data in subsequent CUs.

For a series of video blocks, such as a slice or frame, filter unit 349may select sets of filters for each input in a manner that promotes thevideo quality. For example, filter unit 349 may select sets of filtersfrom pre-defined sets of coefficients, or may adaptively define filtersin order to promote video quality or improved compression. Filter unit349 may select or define one or more sets of filters for a given CU suchthat the same set(s) of filters are used for pixels of different videoblocks of that CU. For a particular frame, slice, or LCU, filter unit349 may apply several sets of filters to multiple inputs, and FSU 353may select the set that produces the best quality video or the highestlevels of compression. Alternatively, FSU 353 may train a new filter byanalyzing the auto-correlations and cross-correlations between multipleinputs and an original image. A new set of filters may, for example, bedetermined by solving Wienter-Hopt equations based on the auto- andcross-correlations. Regardless of whether a new set of filters istrained or an existing set of filters are selected, filter unit 349generates syntax for inclusion in the bitstream that enables a decoderto also identify the set or sets of filters to be used for theparticular frame or slice.

According to this disclosure, for each pixel of a CU within the seriesof video blocks, filter unit 349 may select which filter from the set offilters is to be used based on two or more metrics that quantifyproperties associated with one or more sets of pixels within the CU. Inthis way, FSU 353 may determine sets of filters for a higher level codedunit such as a frame or slice, while filter unit 349 determines whichfilter(s) from the set(s) is to be used for a particular pixel of alower level coded unit based on the two or more metrics associated withthe pixels of that lower level coded unit.

A set of M filters may be used for each input. Depending on designpreferences, M may, for example, be as few as 2 or as great as 16, oreven higher. A large number of filters per input may improve videoquality, but also may increase overhead associated with signaling setsof filters from encoder to decoder. The set of M filters can bedetermined by FSU 353 as described above and signaled to the decoder foreach frame or slice. A segmentation map can be used to indicate how a CUis segmented and whether or not a particular sub-unit of the CU is to befiltered. The segmentation map, may for example, include for a CU anarray of split flags as described above as well an additional bitsignaling whether each sub-CU is to be filtered. For each inputassociated with a pixel of a CU that is to be filtered, a specificfilter from the set of filters can be chosen based on two or moremetrics. Combinations of values for two or more metrics can be indexedto particular filters from the set of M filters.

FIG. 4A is a conceptual diagram illustrating ranges of values for twometrics indexed to filters from a set of filters. The particular exampleof FIG. 4A shows eight filters (i.e. Filter 1, Filter 2 . . . Filter 8),but more or fewer filters may similarly be used. FIG. 4A shows twometrics that might be used for selecting a filter in accordance with thetechniques of this disclosure. The two metrics may, for example,quantify properties of the pixel data related to non-direction specificactivity (e.g. a sum-modified Laplacian value) and direction,direction-specific activity and edge detection, a direction metric andan edge metric, a horizontal activity metric and a vertical activitymetric, or two other such metrics. In some instances, three or moremetrics might be used, in which case the conceptual diagram of FIG. 4Awould include a third dimension for mapping ranges of the metrics tofilters from the set of filters.

In the example of FIG. 4A, a first metric (Metric 1) has four ranges(Ranges 1-1, 1-2, 1-3, and 1-4), and a second metric (Metric 2) also hasfour ranges (Ranges 2-1, 2-2, 2-3, and 2-4). Therefore, the example ofFIG. 4A has sixteen combinations of ranges for Metric 1 and Metric 2. Ascan be seen from FIG. 4A, however, each combination is not necessarilyassociated with a unique filter. The combination of Range 1-1 and Range2-1, as well as combinations 1-1 and 2-2, and 1-1 and 2-3, for instance,are all mapped to Filter 1, in the example of FIG. 4A. Filter 4, incontrast, is only mapped to one combination (1-1 and 2-4). Although theranges of FIG. 4A are shown as being relatively equal, the sizes ofranges may vary. For example, in some implementations, Range 1-1 mayencompass a greater range of values than Range 1-2. Additionally,although FIG. 4A shows Metric 1 and Metric 2 as having the same numberof ranges, the number of ranges for a first metric and the number ofranges for a second metric do not necessarily need to be equal. If, forexample, Metric 1 is a variance metric and Metric 2 is a directionmetric, Metric 1 might use eight ranges while Metric 2 uses threeranges.

In some examples, the ranges of Metric 1 and Metric 2 may represent acontinuous spectrum of values. For example, if Metric 1 is asum-modified Laplacian value, Range 1-2 may correspond to more activitythan Range 1-1 but less activity than Range 1-3, and Range 1-4 maycorrespond to more activity than Range 1-3. Within a range, the amountof activity determined for a particular pixel or group of pixels maysimilarly increase along the Metric 1 axis. In other examples, theranges of Metric 1 and Metric 2 may not represent actual ranges butinstead may represent discrete determinations. For example, if Metric 2is a direction metric, Range 1-1 may correspond to a determination of nodirection, Range 2-2 may correspond to a determination of horizontaldirection, Range 2-3 may correspond to a determination of verticaldirection, and Range 2-4 may represent a determination of diagonaldirection. As will be described in more detail below, no direction,horizontal direction, vertical direction, and diagonal direction can bediscrete determinations, and thus, the ranges for Metric 2 might notrepresent a continuous spectrum of values in the same way the ranges ofMetric 1 do.

FIG. 4B is a conceptual diagram illustrating ranges of values for anactivity metric and a direction metric. In the example of FIG. 4B, thedirection metric includes three discrete determinations (No Direction,Horizontal, and Vertical). Techniques for determining no direction,horizontal, and vertical as well as techniques for determining activitywill be explained in greater detail below. The particular example ofFIG. 4B shows six filters (i.e. Filter 1, Filter 2 . . . Filter 6), butmore or fewer filters may similarly be used. As can be seen by FIG. 4B,the two metrics (activity and direction) create 15 combinations,identified as combinations 421 through 435. In some instances, however,additional combinations not explicitly shown in FIG. 4B may also beused. For example, a combination corresponding to no activity may be a16th combination that also has a corresponding filter.

Filter unit 349 can store a mapping of filters to combinations of rangesof two or more metrics, such as the example mappings of FIGS. 4A and 4B,and use the mapping to determine which filter from a set of filters toapply to a particular pixel or group of pixels in a CU. The mapping offilters to combinations of ranges of two or more metrics may, forexample, be determined by filter unit 349 as part of the filterselection process described above. Regardless of how the mapping isdetermined, filter unit 349 can generate information allowing a decoderto reconstruct the mapping. This information can be included in thecoded bitstream to signal the mapping of combinations of ranges tofilters. The mapping of combinations to ranges signaled may map rangecombinations to filter identifications IDs. The actual coefficients fora particular filter might be signaled separately.

In order to generate this information, filter unit 349 first determinesa transmission order for the combinations. The transmission ordergenerally refers to the order in which filters will be signaled forcombinations of ranges. Using FIG. 4A as an example, filter unit 349might use a left-to-right, top-to-bottom transmission order where thefilter for combination 401 is signaled first, the filter for combination402 is signaled second, and the remaining combinations are signaled inthe order of403=>404=>405=>406=>407=>408=>409=>410=>411=>412=>413=>414=>415=>416.Filter unit 349 might also use a top-to-bottom, zig-zag transmissionorder where the filters for combinations are signaled in the order of401=>402=>403=>404=>408=>407=>406=>405=>409=>410=>411=>412=>416=>415=>414=>413.Filter unit 349 might also use a top-to-bottom, left-to-righttransmission order where the filters for combinations are signaled inthe order of401=>405=>409=>413=>402=>406=>410=>414=>403=>407=>411=>415=>404=>408=>412=>416.Filter unit 349 might also use a left-to-right, zig-zag transmissionorder where the filters for combinations are signaled in the order of401=>405=>409=>413=>414=>410=>406=>402=>403=>407=>411=>415=>416=>412=>408=>404.Referring to FIG. 4B, filter unit 349 may use a left-to-right,bottom-to-top transmission order such that the transmission order is421=>422=>423=>424=>425=>426=>427=>428=>429=>430=>431=>432=>433=>434=>435.As can be imagined, these are just a few of the many transmission ordersthat are possible.

According to a technique of this disclosure, filter unit 349 can use aseries of codewords to signal the mapping to a decoder. For example,filter unit 349 can generate a first codeword to indicate if a currentcombination being decoded maps to the same filter as the most recentlydecoded combination that shares the same range for the first metric. Ifa current combination being decoded maps to the same filter as the mostrecently decoded combination that shares the same range for the secondmetric, then filter unit 349 can generate a second codeword instead ofthe first codeword. If a current combination being decoded does not mapto the same filter as either of these most recently decodedcombinations, then filter unit 349 can generate a third codeword,instead of the first codeword or second codeword, that indicates thefilter corresponding to the current combination being decoded. The firstand second codeword of the current example may be relatively shortcompared to the third codeword. For example, the first codeword andsecond codeword might each be two bits (e.g. 00 and 01, respectively),while the third codeword is more bits (a first bit of 1, plus additionalbits). In this particular context, a current combination being decodedor a previous combination being decoded refers to the portion of theencoding and decoding processes where the mapping of filters to rangecombinations is being signaled by an encoder or constructed by adecoder, and not necessarily to a transmission or decoding of thecombination itself.

Examples of the techniques described above will now be given withreference to FIG. 4A and a top-to-bottom, left-to-right transmissionorder. If, for example, combination 407 is the combination currentlybeing decoded, then combination 406 is the most recently decodedcombination that shares the same range for Metric 1, and combination 403is the most recently decoded combination that shares the same range forMetric 2. If combination 407 maps to the same filter (Filter 7 in FIG.4A) as the most recently decoded combination that shares the same rangefor a second metric (i.e. Range 2-3 for Metric 2), then filter unit 349can transmit a second codeword (e.g. 01) to indicate that the currentcombination being decoded (combination 407) maps to the same filter asthe most recently decoded combination that shares the same range for asecond metric (combination 403).

If, for example, combination 410 is the current combination beingdecoded, then combination 409 is the most recently decoded combinationthat shares the same range for Metric 1, and combination 406 is the mostrecently decoded combination that shares the same range for Metric 2. Ifcombination 410 maps to the same filter (Filter 2 in FIG. 4A) as themost recently decoded combination that shares the same range for a firstmetric (i.e. Range 1-2 for Metric 1), then filter unit 349 can transmita first codeword (e.g. 00) to indicate that the current combinationbeing decoded (combination 410) maps to the same filter (Filter 2) asthe most recently decoded combination that shares the same range for afirst metric (combination 409).

If, for example, combination 411 is the current combination beingdecoded, then combination 410 is the most recently decoded combinationthat shares the same range for Metric 1, and combination 407 is the mostrecently decoded combination that shares the same range for Metric 2. Ifcombination 411 does not map to the same filter as either of combination410 or combination 407, then filter unit 349 can transmit a thirdcodeword (e.g. 1+additional bits) to indicate that the currentcombination being decoded (combination 411) maps to a different filter(Filter 3) than both the most recently decoded combination that sharesthe same range for Metric 1 and the most recently decoded combinationthat shares the same range for Metric 2.

For those current combinations where a combination that shares the samerange for Metric 1 or a combination that shares the same range forMetric 2 have not yet been decoded, then those options can either beconsidered unavailable or can be replaced by a different combination.If, for example, combination 409 is the current combination to bedecoded, then combination 405 is the most recently decoded combinationthat shares the same range for Metric 2, but no combination that sharesa range for Metric 1 has yet been decoded. In such instances, the mostrecently decoded combination that shares a range for Metric 1 can beassumed to not map to the same filter as the current combination beingdecoded. Thus, in this case, the first codeword will not be used forcombination 409. Alternatively, the combination that shares a range forMetric 1 can be replaced by another combination, such as the mostrecently decoded combination or a different previously decodedcombination. In such an instance, the most recently decoded combinationbefore combination 409 would be combination 408. Thus, if combination408 maps to the same filter as combination 409, then filter unit 349 cangenerate the first codeword. Analogous techniques can be used for thosecombinations where a previous combination sharing common range forMetric 1 have not yet been decoded.

For the first combination in a transmission order (e.g. combination 401in the example of FIG. 4A), where neither a combination that shares thesame range for Metric 1 or a combination that shares the same range forMetric 2 have been decoded, filter unit 349 can generate a codewordindicating the filter that maps to the first combination. The filtermay, for example, be signaled using the third codeword or may besignaled using a different technique, in which case the techniquesdescribed in this disclosure might begin with the second combination ina transmission order or a later combination.

According to another technique of this disclosure, filter unit 349 canuse a series of codewords to signal the mapping to a decoder. In someimplementations, filter unit 349 can generate a first codeword toindicate if a current combination being decoded maps to the same filteras the most recently decoded combination that shares the same range forthe first metric. If a current combination being decoded does not map tothe same filter as the most recently decoded combination that sharesthat range for the first metric, then filter unit 349 can generate asecond codeword, instead of the first codeword, that indicates thefilter that maps to the current combination being decoded. In thisexample, the first codeword may be relatively short compared to thesecond codeword. For example, the first codeword might be one bits (e.g.0), while the second codeword is more bits (e.g., a first bit of 1, plusadditional bits). Unlike the previous technique where a short codewordmight be generated if a current combination maps to the same filter as apreviously decoded combination that shares the same range for eitherMetric 1 or Metric 2, this technique includes only generating a shortcodeword if the current combination maps to the same filter as apreviously decoded combination that shares the same range for Metric 1.Thus, even if the current combination maps to the same filter as apreviously decoded combination that shares the same range for Metric 2,filter unit 349 still generates a second codeword (e.g. 1+additionalbits). Although this disclosure is using Metric 1 for purposes ofexplanation, the same techniques can also be applied using only Metric2.

According to yet another technique of this disclosure, filter unit 349can use a different series of codewords to signal the mapping to adecoder. For example, filter unit 349 can generate a first codeword toindicate if a current combination being decoded maps to the same filteras the most recently decoded combination, regardless of which, if any,range the current combination has in common with the previously decodedcombination. If the current combination being decoded does not map tothe same filter as the most recently decoded combination, then filterunit 349 can generate a second codeword identifying the filter that mapsto the current combination. In this particular implementation, the firstcodeword may be relatively short compared to the second codeword. Forexample, the first codeword might be one bits (e.g. 0), while the secondcodeword is more bits (e.g., a first bit of 1, plus additional bits).

Again, using the example of FIG. 4A and a top-to-bottom, left-to-righttransmission order, combination 401 would be the most recently decodedcombination if combination 402 is currently being decoded, combination402 would be the most recently decoded combination if combination 403 isthe current combination, and so on. 404 would be the most recentlydecoded combination if combination 405 is the current combination beingdecoded. Thus, filter unit 349 can generate the first codeword ifcombination 402 maps to the same filter as combination 401, ifcombination 403 maps to the same filter as combination 402, etc.Otherwise, filter unit 349 can generated the second codeword identifyingthe filter that maps to the current combination.

According to yet another technique of this disclosure, filter unit 349can use two codewords to signal the mapping of the filters tocombinations. A first codeword, such as a “0”, can be used to signalthat a current combination uses the same filter as a previouscombination. A second codeword, such as a “1”, can be used to signalthat a current combination has a different filter than the previouscombination. The second codeword, however, does not need to identify anew filter. Instead, the new filter can be determined based on thetransmission order for the classes and the order in which filtercoefficients are transmitted. Using the left-to-right, bottom-to-toptransmission order described above for FIG. 4B as an example, codewordsmight be transmitted accordingly: 421 (0)=>422 (0)=>423 (1)=>424(0)=>425 (0)=>426 (0)=>427 (0)=>428(1)=>429 (0)=>430 (0)=>431 (0)=>432(1)=>433 (0)=>434 (0)=>435 (0), with the number in parenthesesrepresenting the codeword for that combination. In this example,combinations 421-422 would be mapped to a first filter, combinations423-427 to a second filter, combinations 428-431 to a third filter, andcombinations 432-435 to a fourth filter. The coefficients for the firstfilter, second filter, third filter, and fourth filter can correspond tothe order in which sets of filter coefficients are signaled, where thefirst set of filter coefficients signaled correspond to the firstfilter, the second set of filter coefficients signaled correspond to thesecond filter, and so on. Determining an order for transmitting sets offilter coefficients is discussed in more detail below.

The various techniques described in this disclosure for signaling amapping of filters to combinations of ranges are not mutually exclusivealternatives, but rather, may be used in conjunction with one another.For example, in some implementations, certain combinations might besignaled using a first technique while other combinations are signaledusing a second technique. As one example, where one of a combinationthat shares the same range for Metric 1 or a combination that shares thesame range for Metric 2 have not yet been decoded (e.g. combinations402, 403, 404, 405, 409, and 413), then filter unit 349 may use a firsttechnique. Where both a combination that shares the same range forMetric 1 and a combination that shares the same range for Metric 2 havebeen decoded (e.g. combinations 406, 407, 408, 410, 411, 412, 414, 415,and 416), then a second technique might be used. Additionally, thecodewords used for any of the first, second, and third codewordsdescribed above may be any of fixed length codewords, variable lengthcodewords, or context-adaptive variable length codewords.

In addition to generating information allowing a decoder to reconstructthe mapping of filters to combinations of ranges, filter unit 349 alsogenerates information allowing a decoder to reconstruct the filtersthemselves. Reconstructing the filters includes reconstructing thefilter coefficients of the filters. As will be described in more detailbelow, filter unit 349 can use differential coding techniques to signalthe filter coefficients. To use differential coding technique, filterunit 349 determines an order in which to signal the sets of filtercoefficients.

As part of determining the order, filter unit 349 determines acombination identification (ID) that represents a sequential value foreach combination of ranges. Using FIG. 4A as an example, thecombinations might be assigned combination IDs that represent sequentialvalues in a left-to-right, top-to-bottom order, in which casecombination 401 would be assigned the first sequential value,combination 402 would be assigned the second sequential value, and theremaining combinations would be assigned sequential values in the orderof 403=>404=>405=>406=>407=>408=>409=>410=>411=>412=>413=>414=>415=>416.Filter unit 349 might also assign the combination IDs using atop-to-bottom, zig-zag order where the combinations would be assignedcombination IDs with sequential values that are in an order of401=>402=>403=>404=>408=>407=>406=>405=>409=>410=>411=>412=>416=>415=>414=>413.Filter unit 349 might also assign combination IDs using a top-to-bottom,left-to-right order where the combinations are assigned combination IDswith sequential values that are in an order of401=>405=>409=>413=>402=>406=>410=>414=>403=>407=>411=>415=>404=>408=>412=>416.Filter unit 349 might also use a left-to-right, zig-zag order where thecombinations are assigned combination IDs with sequential values in anorder of401=>405=>409=>413=>414=>410=>406=>402=>403=>407=>411=>415=>416=>412=>408=>404.As can be imagined, these are just a few of the many orders that couldbe used. Furthermore, any of the orders described could be either lowestto highest or highest to lowest.

After filter unit 349 has determined the mapping of filters to rangecombinations, filter unit 349 can identify groupings of rangecombinations that are mapped to the same filter. Using FIG. 4A as anexample, the groupings would be as follows.

Filter 1 Group: combinations 413, 414, and 415

Filter 2 Group: combinations 409, 410

Filter 3 Group: combinations 411 and 412

Filter 4 Group: combination 416

Filter 5 Group: combinations 401 and 405

Filter 6 Group: combinations 402 and 406

Filter 7 Group: combinations 403 and 407

Filter 8 Group: combinations 404 and 408.

Filter unit 349 can then assign each group a group ID, and the group IDcan represent a sequential value. The group IDs can be assigned to thegroups based on the sequential values associated with the combinationsthat comprise the group. For example, the group that has the combinationwith the lowest associated sequential value based on the combinationIDs, might be assigned the group ID with the lowest sequential value. Ofthe remaining groups, the remaining group that has the combination withthe lowest associated sequential value can be assigned the group ID withthe next lowest sequential value. This process can repeat until allgroups have been assigned a group ID. In some implementations, group IDsmight be assigned based on the combinations with the highest associatedsequential values rather than the lowest. In some implementations, thegroup that has the combination with the lowest associated sequentialvalue based on the combination IDs, might be assigned the group ID withthe highest sequential value, or vice versa.

Again, using FIG. 4A as an example, and assuming that combinations401-416 are assigned combination IDs with sequential values in aleft-to-right, top-to-bottom order, then filter unit 349 can assigngroup IDs to the filter groups, as shown below in Table 1.

TABLE 1 Combinations Combination with Group Name in group lowestsequential value Group ID Filter 1 Group 413, 414, 415 413 7 Filter 2Group 409, 410 409 5 Filter 3 Group 411, 412 411 6 Filter 4 Group 416416 8 Filter 5 Group 401, 405 401 1 Filter 6 Group 402, 406 402 2 Filter7 Group 403, 407 403 3 Filter 8 Group 404, 408 404 4

In the example of FIG. 4A, shown in Table 1, filter unit 349 assigns theFilter 5 Group the group ID with the lowest sequential value because theFilter 5 Group includes the range combination with the lowest sequentialvalue (i.e., combination 401). Filter unit 349 assigns the Filter 6Group the group ID with the second lowest sequential value because, ofthe remaining filter groups (i.e. all the groups excluding the Filter 5Group), the Filter 6 Group includes the range combination with thesecond lowest sequential value (i.e., combination 402). Filter unit 349assigns the Filter 7 Group the group ID with the third lowest sequentialvalue because, of the remaining filter groups (i.e. all the filtergroups excluding the Filter 5 Group and the Filter 6 Group), the Filter7 Group includes the range combination with the lowest sequential value(i.e., combination 403). Filter unit 349 assigns the Filter 8 Group thegroup ID with the fourth lowest sequential value because, of theremaining filter groups (i.e. all the filter groups excluding the Filter5 Group, the Filter 6 Group, and the Filter 7 Group), the Filter 8 Groupincludes the range combination with the fourth lowest sequential value(combination 404). Filter unit 349 assigns the Filter 2 Group the groupID with the fifth lowest sequential value because, of the remainingfilter groups (i.e. excluding the Filter 5 Group, the Filter 6 Group,the Filter 7 Group, and the Filter 8 Group), the Filter 2 Group includesthe range combination with the lowest sequential value (combination409). Filter unit 349 assigns the Filter 3 Group the group ID with thesixth lowest sequential value because, of the remaining filter groups(i.e. excluding the Filter 5 Group, the Filter 6 Group, the Filter 7Group, the Filter 8 Group, and the Filter 2 Group), the Filter 3 Groupincludes the range combination with the lowest sequential value(combination 411). Filter unit 349 assigns the Filter 1 Group the groupID with the seventh lowest sequential value because, of the remainingfilter groups (i.e. excluding the Filter 5 Group, the Filter 6 Group,the Filter 7 Group, the Filter 8 Group, the Filter 2 Group, and theFilter 3 Group), the Filter 1 Group includes the range combination withthe lowest sequential value (combination 413). Finally, filter unit 349assigns the Filter 4 group, the final remaining filter group, the groupID with the highest sequential value (8 in this particular example).

Based on the filter group IDs, filter unit 349 determines an order inwhich to signal the filter coefficients of a filter. Again, using theexample of FIG. 4A and Table 1, filter unit 349 first signals thecoefficient for Filter 5, then the coefficient for Filter 6, then thecoefficient for Filter 7, then the coefficient for Filter 8, then thecoefficient for Filter 2, then the coefficient for Filter 3, then thecoefficient for Filter 1, and finally the coefficient for Filter 4.Using differential coding techniques, as described in this disclosure,filter unit 349 may code the coefficients for Filter 6 as differenceinformation relative to the filter coefficients of Filter 5, code thecoefficients for Filter 7 as difference information relative to thefilter coefficients for Filter 6, and so on, based on the sequentialordering of Group IDs.

The mapping of two or more metrics for inputs to filters can beimplemented in multiple ways. For example, in some implementations eachinput might have a unique set of filters, while in some implementationsinputs share a common set of filters. Additionally, in someimplementations, two or more metrics for each input might be used toidentify a particular filter for each input. In other implementations,however, two or more metrics for a single input might be used toidentify filters for all the inputs. In yet other implementations, twoor more metrics for a first input might be used to identify a filter fora second, different input.

In accordance with this disclosure, filter unit 349 may perform codingtechniques with respect to filter information that may reduce the amountof data needed to encode and convey filter information from encoder 350to another device. Again, for each frame or slice, filter unit 349 maydefine or select one or more sets of filter coefficients to be appliedto the pixels of CUs for that frame or slice. Filter unit 349 appliesthe filter coefficients in order to filter video blocks of reconstructedvideo frames stored in memory 334, which may be used for predictivecoding consistent with in-loop filtering. Filter unit 349 can encode thefilter coefficients as filter information, which is forwarded to entropyencoding unit 346 for inclusion in the encoded bitstream.

Additionally, the techniques of this disclosure may exploit the factthat some of the filter coefficients defined or selected by FSU 353 maybe very similar to other filter coefficients applied with respect to thepixels of CUs of another frame or slice. The same type of filter may beapplied for different frames or slices (e.g., the same filter support),but the filters may be different in terms of filter coefficient valuesassociated with the different indices of the filter support.Accordingly, in order to reduce the amount of data needed to convey suchfilter coefficients, filter unit 349 may predictively encode one or morefilter coefficients to be used for filtering based on the filtercoefficients of another CU, potentially exploiting similarities betweenthe filter coefficients. In some cases, however, it may be moredesirable to encode the filter coefficients directly, e.g., withoutusing any prediction. Various techniques, such as techniques thatexploit the use of an activity metric to define when to encode thefilter coefficients using predictive coding techniques and when toencode the filter coefficients directly without any predictive coding,can be used for efficiently communicating filter coefficients to adecoder. Additionally, symmetry may also be imposed so that a subset ofcoefficients (e.g., 5, −2, 10) known by the decoder can be used todefine the full set of coefficients (e.g., 5, −2, 10, 10, −2, 5).Symmetry may be imposed in both the direct and the predictive codingscenarios.

As described above, video encoder 350 represents an example of a videoencoder configured to determine a first metric for a group of pixelswithin a block of pixels, determine a second metric for the group ofpixels, determine a filter based on the first metric and the secondmetric, and generate a filtered image by applying the filter to thegroup of pixels. Video encoder 350 also represents an example of a videoencoder configured to determine a first metric for a block of pixels,wherein the first metric is determined based on a comparison of a subsetof the pixels in the block to other pixels in the block; determine asecond metric for the block of pixels; determine a filter based on thefirst metric and the second metric; and, generate a filtered image byapplying the filter to the block of pixels.

As described above, video encoder 350 also represents an example of avideo encoder configured to determine a mapping of range combinations tofilters, wherein a range combination comprises a range for a firstmetric and a range for a second metric, wherein each range combinationhas a unique range combination identification (ID), wherein each uniquerange combination ID corresponds to a sequential value for a rangecombination; assign unique group IDs to groups of range combinationsbased on the sequential values for the range combinations, wherein eachunique group ID corresponds to a sequential value for a group; and, codesets of filter coefficients corresponding for the filters based on theunique group IDs. Video encoder 350 can code the sets of filtercoefficients by signaling the sets of filter coefficients in a codedbitstream in an order that is selected based on the sequential values ofthe unique group IDs. Video encoder 350 can signal the sets of filtercoefficients using differential coding techniques.

As described above, video encoder 350 also represents an example of avideo encoder configured to determine a mapping of range combinations tofilters, wherein a range combination comprises a range of values for afirst metric and a range of values for a second metric; generate a firstcodeword if a current range combination is mapped to the same filter asa previous range combination that comprises the same range of values forthe first metric; generate a second codeword if a current rangecombination is mapped to the same filter as a previous range combinationthat comprises the same range of values for the second metric; and,generate a third codeword if the current range combination is mapped toa different filter than the previous range combination that comprisesthe same range of values for the first metric and the previous rangecombination that comprises the same range of values for the secondmetric. Video encoder 350 also represents an example of a video encoderconfigured to determine a mapping of range combinations to filters,wherein a range combination comprises a range for a first metric and arange for a second metric; generate a first codeword if a current rangecombination is mapped to the same filter as a previous rangecombination; and, generate a second codeword if the current rangecombination is mapped to a different filter than the previous rangecombination, wherein the second codeword identifies a filter mapped tothe current range combination.

FIG. 5 is a block diagram illustrating an example of a video decoder560, which decodes a video sequence that is encoded in the mannerdescribed herein. The received video sequence may comprise an encodedset of image fames, a set of frame slices, a commonly coded group ofpictures (GOPs), or a wide variety of types of series of video blocksthat include encoded video blocks and syntax to define how to decodesuch video blocks.

Video decoder 560 includes an entropy decoding unit 552, which performsthe reciprocal decoding function of the encoding performed by entropyencoding unit 346 of FIG. 3. In particular, entropy decoding unit 552may perform CAVLC or CABAC decoding, or any other type of entropydecoding used by video encoder 350. Entropy decoded video blocks in aone-dimensional serialized format may be inverse scanned to convert oneor more one-dimensional vectors of coefficients back into atwo-dimensional block format. The number and size of the vectors, aswell as the scan order defined for the video blocks may define how thetwo-dimensional block is reconstructed. Entropy decoded predictionsyntax may be sent from entropy decoding unit 552 to prediction module554, and entropy decoded filter information may be sent from entropydecoding unit 552 to filter unit 559.

Video decoder 560 also includes a prediction module 554, an inversequantization unit 556, an inverse transform unit 558, a memory and asummer 564. In addition, video decoder 560 also includes a de-blockingfilter 557 that filters the output of summer 564. Consistent with thisdisclosure, filter unit 559 may receive entropy decoded filterinformation that includes one or more filters to be applied to one ormore inputs. Although not shown on FIG. 5, de-blocking filter 557 mayalso receive entropy decoded filter information that includes one ormore filters to be applied.

The filters applied by filter unit 559 may be defined by sets of filtercoefficients. Filter unit 559 may be configured to generate the sets offilter coefficients based on the filter information received fromentropy decoding unit 552. The filter information may include filterdescription syntax that identifies a maximum number of filters in a setof filters and/or a shape of filters in a set of filters, for example.The filter description syntax can be included in a header of a series ofvideo blocks, e.g., an LCU header, a frame header, a slice header, a GOPheader, a sequence header, or the like. In other examples, the filterdescription syntax might be included in a footer or other datastructure. Based on the filter description syntax, filter unit 559 canreconstruct the set of filters used at the encoder.

The filter information may also include additional signaling syntax thatsignals to the decoder the manner of encoding used for any given set ofcoefficients. In some implementations, the filter information may forexample, also include ranges for two or more metrics for which any givenset of coefficients should be used. Following decoding of the filters,filter unit 559 can filter the pixel values of decoded video blocksbased on the one or more sets of filter coefficients and the signalingsyntax that includes the ranges for which the different sets of filtercoefficients should be used.

Filter unit 559 may receive in the bitstream one or more syntax elementsindicating a set of filters for each frame or slice as well as a mappingof filters to the two or more metrics. For example, if an encoder usesthe mapping of ranges for metrics to filters shown in FIG. 4A, then theencoder will either signal this mapping or transmit data to allow filterunit 559 to reconstruct this mapping. Regardless of whether or not thismapping is explicitly signaled, filter unit 559 can maintain the samemapping of filters to combinations of ranges as used by the encoder.

As mentioned above, filter unit 559 generates a mapping based on filterinformation signaled in the bitstream. Based on this mapping, filterunit 559 can determine groups and assign group IDs to the groups in thesame manner described above in relation to filter unit 349. Using thesegroup IDs, filter unit 559 can associate received filter coefficientswith

For each CU within the frame or slice, filter unit 559 can calculate oneor more metrics associated with the decoded pixels of a CU for multipleinputs (i.e. PI, EI, pRI, and RI) in order to determine which filter(s)of the set(s) to apply to each input. Alternatively, filter unit 559 maycalculate one or more metrics for a single input, such as pRI or RI.Filter unit 559 determines which filter to apply based on the metricsdetermined for a particular pixel or group of pixels. Using asum-modified Laplacian value and direction as examples for Metric 1 andMetric 2 and using the mappings shown in FIG. 4A as an example, iffilter unit 559 determines that a pixel or group of pixels has asum-modified Laplacian value in Range 1-2 and a direction correspondingto Range 2-3, then filter unit 559 can apply Filter 2 to that pixel orgroup of pixels. If filter unit 559 determines that a pixel or group ofpixels has a sum-modified Laplacian value in Range 1-4 and a directioncorresponding to Range 2-2, then filter unit 559 can apply Filter 6 tothat pixel or group of pixels, and so on. The filter may generallyassume any type of filter support shape or arrangement. The filtersupport refers to the shape of the filter with respect to a given pixelbeing filtered, and the filter coefficients may define weighting appliedto neighboring pixel values according to the filter support. Accordingto the techniques of the present disclosure, syntax data may be includedin the bitstream to signal to the decoder how the filters were encoded(e.g., how the filter coefficients were encoded), as well as the rangesof the activity metric for which the different filters should be used.

For each CU within the frame or slice, filter unit 559 can calculate oneor more metrics associated with the decoded pixels of a CU for multipleinputs (i.e. PI, EI, pRI, and RI) in order to determine which filter(s)of the set(s) to apply to each input. Alternatively, filter unit 559 maycalculate one or more metrics for a single input, such as pRI or RI.Filter unit 559 determines which filter to apply based on the metricsdetermined for a particular pixel or group of pixels. Using asum-modified Laplacian value and direction as examples for Metric 1 andMetric 2 and using the mappings shown in FIG. 4A as an example, iffilter unit 559 determines that a pixel or group of pixels has asum-modified Laplacian value in Range 1-2 and a direction correspondingto Range 2-3, then filter unit 559 can apply Filter 2 to that pixel orgroup of pixels. If filter unit 559 determines that a pixel or group ofpixels has a sum-modified Laplacian value in Range 1-4 and a directioncorresponding to Range 2-2, then filter unit 559 can apply Filter 6 tothat pixel or group of pixels, and so on. The filter may generallyassume any type of filter support shape or arrangement. The filtersupport refers to the shape of the filter with respect to a given pixelbeing filtered, and the filter coefficients may define weighting appliedto neighboring pixel values according to the filter support. Accordingto the techniques of the present disclosure, syntax data may be includedin the bitstream to signal to the decoder how the filters were encoded(e.g., how the filter coefficients were encoded), as well as the rangesof the activity metric for which the different filters should be used.

Prediction module 554 receives prediction syntax (such as motionvectors) from entropy decoding unit 552. Using the prediction syntax,prediction module 554 generates the prediction blocks that were used tocode video blocks. Inverse quantization unit 556 performs inversequantization, and inverse transform unit 558 performs inverse transformsto change the coefficients of the residual video blocks back to thepixel domain. Adder 564 combines each prediction block with thecorresponding residual block output by inverse transform unit 558 inorder to reconstruct the video block.

Filter unit 559 generates the filter coefficients to be applied for eachinput of a CU, and then applies such filter coefficients in order tofilter the reconstructed video blocks of that CU. The filtering, forexample, may comprise additional deblock filtering that smoothes edgesand/or eliminates artifacts associated with video blocks, denoisefiltering to reduce quantization noise, or any other type of filteringthat can improve coding quality. The filtered video blocks areaccumulated in memory 562 in order to reconstruct decoded frames (orother decodable units) of video information. The decoded units may beoutput from video decoder 560 for presentation to a user, but may alsobe stored for use in subsequent predictive decoding.

In the field of video coding, it is common to apply filtering at theencoder and decoder in order to enhance the quality of a decoded videosignal. Filtering can be applied via a post-filter, in which case thefiltered frame is not used for prediction of future frames.Alternatively, filtering can be applied “in-loop,” in which case thefiltered frame may be used to predict future frames. A desirable filtercan be designed by minimizing the error between the original signal andthe decoded filtered signal. Typically, such filtering has been based onapplying one or more filters to a reconstructed image. For example, adeblocking filter might be applied to a reconstructed image prior to theimage being stored in memory, or a deblocking filter and one additionalfilter might be applied to a reconstructed image prior to the imagebeing stored in memory.

In a manner similar to the quantization of transform coefficients, thecoefficients of the filter h(k,l), where k=−K, . . . , K, and l=−L, . .. , L may also be quantized. K and L may represent integer values. Thecoefficients of filter h(k,l) may be quantized as:

f(k,l)=round(normFact·h(k,l))

where normFact is a normalization factor and round is the roundingoperation performed to achieve quantization to a desired bit-depth.Quantization of filter coefficients may be performed by filter unit 349of FIG. 3 during the encoding, and de-quantization or inversequantization may be performed on decoded filter coefficients by filterunit 559 of FIG. 5. Filter h(k,l) is intended to generically representany filter. For example, filter h(k,l) could be applied to any one ofmultiple inputs. In some instances multiple inputs associated with avideo block will utilize different filters, in which case multiplefilters similar to h(k,l) may be quantized and de-quanitzed as describedabove.

The quantized filter coefficients are encoded and sent from sourcedevice associated with encoder 350 to a destination device associatedwith decoder 560 as part of an encoded bitstream. In the example above,the value of normFact is usually equal to 2n although other values couldbe used. Larger values of normFact lead to more precise quantizationsuch that the quantized filter coefficients f(k, l) provide betterperformance. However, larger values of normFact may produce coefficientsf(k, l) that require more bits to signal to the decoder.

At decoder 560 the decoded filter coefficients f(k,l) may be applied tothe appropriate input. For example, if the decoded filter coefficientsare to be applied to RI, the filter coefficients may be applied to thepost-deblocked reconstructed image RI(i,j), where i=0, . . . , M andj=0, . . . , N as follows:

${\overset{\sim}{R}{I\left( {i,j} \right)}} = {\sum\limits_{k = {- K}}^{K}\; {\sum\limits_{l = {- L}}^{L}\; {{f\left( {k,l} \right)}{{{RI}\left( {{i + k},{j + l}} \right)}/{\sum\limits_{k = {- K}}^{K}\; {\sum\limits_{l = {- L}}^{L}\; {f\left( {k,l} \right)}}}}}}}$

The variables M, N, K and L may represent integers. K and L may define ablock of pixels that spans two-dimensions from −K to K and from −L to L.Filters applied to other inputs can be applied in an analogous manner.

The techniques of this disclosure may improve the performance of apost-filter or in-loop filter, and may also reduce number of bits neededto signal filter coefficients f(k, l). In some cases, a number ofdifferent post-filters or in-loop filters are signaled to the decoderfor each series of video block, e.g., for each frame, slice, portion ofa frame, group of frames (GOP), or the like. For each filter, additionalinformation is included in the bitstream to identify the CUs,macroblocks and/or pixels for which a given filter should be applied.

The frames may be identified by frame number and/or frame type (e.g.,I-frames, P-frames or B-frames). I-frames refer to intra-frames that areintra-predicted. P-frames refer to predictive frames that have videoblocks predicted based on one list of data (e.g., one previous frame).B-frames refer to bidirectional predictive frames that are predictedbased on two lists of data (e.g., a previous and subsequent frame).Macroblocks can be identified by listing macroblock types and/or rangeof quantization parameter (QP) values use to reconstruct the macroblock.

Filter coefficients f(k,l), for any input, may be coded using predictionfrom coefficients signaled for previous CUs. For each input of a CU m(e.g., each frame, slice or GOP), the encoder may encode and transmit aset of M filters:

g _(i) ^(m), wherein i=0, . . . ,M−1.

For each filter, the bitstream may also be encoded to identify thecombination of ranges for two or more metrics for which the filtershould be used.

The filter coefficients can be predicted using reconstructed filtercoefficients used in a previous CU. The previous filter coefficients maybe represented as:

f _(i) ^(n) where i=0, . . . ,N−1,

In this case, the number of the CU n may be used to identify one or morefilters used for prediction of the current filters, and the number n maybe sent to the decoder as part of the encoded bitstream. In addition,information can be encoded and transmitted to the decoder to identifycombinations of ranges for two or more metrics for which predictivecoding is used.

The amplitude of the filter coefficients g(k, l) depends on k and lvalues. Usually, the coefficient with the biggest amplitude is thecoefficient g(0,0). The other coefficients which are expected to havelarge amplitudes are the coefficients for which value of k or l is equalto 0. This phenomenon may be utilized to further reduce amount of bitsneeded to signal the coefficients. The index values k and l may definelocations within a known filter support.

The coefficients:

g _(i) ^(m)(k,l),i=0, . . . ,M−1

for each frame m may be coded using parameterized variable length codessuch as Golomb or exp-Golomb codes defined according to a parameter p.By changing the value of parameter p that defines the parameterizedvariable length codes, these codes can be used to efficiently representwide range of source distributions. The distribution of coefficientsg(k,l) (i.e., their likelihood to have large or small values) depends onvalues of k and l. Hence, to increase coding efficiency, for each framem, the value of parameter p is transmitted for each pair (k,l). Theparameter p can be used for parameterized variable length coding whenencoding coefficients:

g _(i) ^(m)(k,l) where k=−K, . . . ,K,l=−L, . . . ,L.

As described above, video decoder 560 represents an example of a videodecoder configured to determine a first metric for a group of pixelswithin a block of pixels, determine a second metric for the group ofpixels, determine a filter based on the first metric and the secondmetric, and generate a filtered image by applying the filter to thegroup of pixels. Video decoder 560 also represents an example of a videoencoder configured to determine a first metric for a block of pixels,wherein the first metric is determined based on a comparison of a subsetof the pixels in the block to other pixels in the block; determine asecond metric for the block of pixels; determine a filter based on thefirst metric and the second metric; and, generate a filtered image byapplying the filter to the block of pixels.

As described above, video decoder 560 also represents an example of avideo decoder configured to determine a mapping of range combinations tofilters, wherein a range combination comprises a range for a firstmetric and a range for a second metric, wherein each range combinationhas a unique range combination identification (ID), wherein each uniquerange combination ID corresponds to a sequential value for a rangecombination; assign unique group IDs to groups of range combinationsbased on the sequential values for the range combinations, wherein eachunique group ID corresponds to a sequential value for a group; and, codesets of filter coefficients corresponding for the filters based on theunique group IDs. Video decoder 560 can code the sets of filtercoefficients comprises by generating the sets of filter coefficientsbased on information received in a coded bitstream. Video decoder 560can generate the sets of filter coefficients using differential codingtechniques.

Video decoder 560 also represents an example of a video decoderconfigured to map a first range combination to a first filter, whereinthe first range combination comprises a first range of values for afirst metric and a first range of values for a second metric; map asecond range combination to a second filter, wherein the second rangecombination comprises a second range of values for the first metric anda second range of values for the second metric; map a current rangecombination to a filter, wherein the current range combination comprisesthe first range of values of the first metric and the second range ofvalues for the second metric. Mapping the current range combination tothe filter can include mapping the current range combination to thefirst filter in response to receiving a first codeword, wherein thefirst codeword indicates the current range combination is mapped to thesame filter as the first range combination; mapping the current rangecombination to the second filter in response to receiving a secondcodeword, wherein the second codeword indicates the current rangecombination is mapped to the same filter as the second combination; and,mapping the current range combination to a third filter in response toreceiving a third codeword, wherein the third codeword identifies thatthird filter. Video decoder 560 also represents an example of a videodecoder configured to generate a mapping of range combinations tofilters, wherein a range combination comprises a range for a firstmetric and a range for a second metric; map a current range combinationto a same filter as a previous range combination in response toreceiving a first codeword signaling the current range combination ismapped to the same filter as the previous range combination; and, mapthe current range combination to a filter identified by a secondcodeword in response to receiving the second codeword signaling thecurrent range combination is mapped to a different filter than theprevious range combination.

As has been introduced above, several different types of metrics can beused in conjunction with the multi-metric filtering techniques describedin this disclosure. Some of these metrics are activity metrics thatquantify activity associated with one or more blocks of pixels withinthe video data. Activity metrics can comprise variance metricsindicative of pixel variance within a set of pixels. As will bedescribed, some of these activity metrics are direction-specific. Forexample, a horizontal activity metric quantifies activity along ahorizontal axis, a vertical activity metric quantifies activity along avertical axis, a diagonal activity metric quantifies activity along adiagonal axis, and so on.

Some activity metrics are not direction-specific. For example, asum-modified Laplacian value is an activity metric based on atwo-dimensional window of pixels that surround a current pixel orcurrent group of pixels. For a current pixel (i,j), a sum-modifiedLaplacian value can be calculated as follows:

$\begin{matrix}{{{var}\left( {i,j} \right)} = {\sum\limits_{k = {- K}}^{K}\; {\sum\limits_{l = {- L}}^{L}\; {{{2{R\left( {{i + k},{j + l}} \right)}} - {R\left( {{i + k - 1},{j + l}} \right)} - {R\left( {{i + k + 1},{j + l}} \right.} + {{{2R\left( {{i + k},{j + l}} \right)} - {R\left( {{i + k},{j + l - 1}} \right)} - {R\left( {{i + k},{j + l + 1}} \right)}}}}}}}} & (1)\end{matrix}$

where k represents a value of a summation of pixel values from −K to Kand l represents a value of a summation from −L to L for atwo-dimensional window that spans from −K to K and −L to L, wherein iand j represent pixel coordinates of the pixel data, RI(i,j) representsa given pixel value at coordinates i and j, and var(i,j) is the activitymetric (i.e. the sum-modified Laplacian value).

The techniques of the present disclosure may also be implemented usingdirection-specific metrics for horizontal activity, vertical activity,and diagonal activity. Equations 2 and 3 show examples of how horizontalactivity and vertical activity can be computed for a current pixel (x,y) by comparing a pixel value (Rec), such as intensity, of the currentpixel to a pixel value of neighboring pixels.

Hor_act(x,y)=R(2*Rec[x][y]−Rec[x+1][y]−Rec[x−1][y])  (2)

Ver_act(x,y)=R(2*Rec[x][y]−Rec[x][y+1]−Rec[x][y+1])  (3)

As shown by equation 2, when determining horizontal activity, thecurrent pixel (x,y) can be compared to a left neighbor (x−1, y) and aright neighbor (x+1, y). As shown by equation 3, when determiningvertical activity, the current pixel can be compared to an upperneighbor (x, y+1) and a lower neighbor (x, y−1).

Equations 4 and 5 show examples of how diagonal activity can be computedfor a current pixel (x, y) by comparing a pixel value (Rec) of thecurrent pixel to pixel values of neighboring pixels.

45deg_act(x,y)=R(2*Rec[x][y]−Rec[x+1][y+1]−Rec[x−1][y−1])  (4)

135deg_act(x,y)=R(2*Rec[x][y]−Rec[x−1][y+1]−Rec[x+1][y−1])  (5)

As shown by equation 4, diagonal activity can be computed, for example,in the 45 degree direction by comparing a current pixel (x, y) to anupper-right neighbor (x+1, y+1) and a lower-left neighbor (x−1, y−1). Asshown by equation 5, diagonal activity may also be in the 135 degreedirection by comparing a current pixel (x, y) to a left-upper neighbor(x−1, y+1) and a right-lower neighbor (x+1, y−1).

Equations 2-5, above, illustrate how horizontal activity, verticalactivity, and diagonal activity can be determined on a pixel-by-pixelbasis, but in some implementations, horizontal activity, verticalactivity, and diagonal activity may be determined on a group-by-groupbasis, where a group of pixels is a 2×2, 4×4, or M×N block of pixels. Insuch an implementation, horizontal activity, for example, can bedetermined by comparing pixel values of a current group to pixel valuesof a left group and a right group, in an analogous manner to equation 2;and, the vertical activity can be determined by comparing a currentgroup to an upper group and a lower group, in an analogous manner toequation 3. Likewise, 45-degree diagonal activity can be determined bycomparing a current group of pixels to an upper-right neighboring groupand a lower-left neighboring group in an analogous manner to equation 4,and 135-degree diagonal activity can be determined by comparing acurrent group of pixels to an upper-left neighboring group and alower-right neighboring group, in an analogous manner to equation 5.

In some implementations, horizontal activity, vertical activity,45-degree diagonal activity, and 135-degree diagonal activity can bedetermined by comparing a current pixel or group of pixels toneighboring pixels or groups of pixels in only one direction. Forexample, instead of determining horizontal activity based on comparing acurrent pixel to a left neighbor and a right neighbor, horizontalactivity might be determined based on only a left neighbor or only aright neighbor. Additionally, in some implementations, horizontalactivity, vertical activity, 45-degree diagonal activity, and 135-degreediagonal activity may be determined using averages or weighted averagesof areas of neighboring pixels instead of single neighboring pixels orsingle groups of pixels.

The values resulting from equations 2-5 can be divided into a finitenumber of ranges, such as 2, 4, 8, or any other finite number, and eachrange can be assigned a range identification. Referring back to FIG. 4A,for example, Range 1-1, Range 1-2, Range 2-1, etc. are all examples ofrange identifications. As one example, horizontal activity values can bedivided into four ranges, and the ranges might be assigned IDs Range1-1, Range 1-2, Range 1-3, and Range 1-4. Horizontal threshold values(i.e., ThH₁, . . . , ThH_(P-1)) can determine where the ranges begin andend. Table 2 below shows the generic case of how horizontal IDs might beassigned to P ranges.

TABLE 2 Index of activity metric Condition of Hor_act_B Horizontal IDHor_act_B < ThH₁ Range 2-1 ThH₁ ≦ Hor_act_B < ThH₂ Range 2-2 . . . . . .ThH_(P−1) ≦ Hor_act_B Range 2-PUsing the example of Table 2, if a current pixel has a horizontalactivity value greater than ThH₁ but less than ThH₂, then the currentpixel is in range 2-2 for metric 2. Current pixels may be assigned tovertical ranges with Vertical IDs, 45-degree diagonal ranges with45-degree diagonal IDS, and 135-degree diagonal ranges with 135-degreediagonal IDs, in a similar manner as described above in Table 2 forhorizontal ranges and horizontal IDs.

Any of horizontal activity, vertical activity, 45-degree diagonalactivity, and 135-degree diagonal activity can be used as a metric inaccordance with the multi-metric filter filtering techniques describedin this disclosure. For example, referring again back to FIG. 4A, Metric1 might be a measure of vertical activity, and Metric 2 might be ameasure of horizontal activity. In such an example, a filter unit, suchas filter unit 349 of FIG. 4A or filter 559 of FIG. 5, can determine afilter for a pixel or group of pixels based on the horizontal activityof the pixel or group of pixel and the vertical activity of the pixel orgroup of pixels. If, for example, a current pixel has a horizontalactivity metric that falls in Range 2-3 and a vertical activity metricthat falls in range 1-3, then the filter unit filters the pixel usingFilter 4. In a similar manner, combinations of 45-degree diagonalactivity and 135-degree diagonal activity, 45-degree diagonal activityand horizontal activity, 45-degree diagonal activity and verticalactivity, 135-degree diagonal activity and horizontal activity, or135-degree diagonal activity and vertical activity may also be used by afilter unit for selecting a filter for a pixel or group of pixels. Insome implementations, three or all four of horizontal activity, verticalactivity, 45-degree diagonal activity, and 135-degree diagonal activitymay be used by a filter unit for selecting a filter of a pixel or groupof pixels.

In the implementations described above, horizontal activity, verticalactivity, 45-degree diagonal activity, and 135-degree diagonal activitycan all be used as metrics, as Metric 1 and/or Metric 2 in FIG. 4A, forexample. In some implementations, however, horizontal activity, verticalactivity, 45-degree diagonal activity, and 135-degree diagonal activitymight not be metrics themselves, but instead can be used as intermediatedeterminations for determining an overall direction metric. Thedirection metric generally describes in which direction (e.g. nodirection, horizontal, vertical, 45-degree diagonal, or 135-degreediagonal) the pixels are changing the most.

In one example, using only horizontal activity and vertical activity asdescribed in equations 2 and 3, a direction for a pixel might bedetermined based on the following conditions:

Direction 1=horizontal, if Hor_activity>k1*Ver_activity

Direction 2=vertical, if Ver_activity>k2*Hor_activity

Direction 0=no direction, otherwise.

Constants, k1 and k2, can be selected such that the direction is onlydeemed to be direction 1 or direction 2 if horizontal activity issubstantially greater than vertical activity or vertical activity issubstantially greater than horizontal activity. If horizontal activityand vertical activity are equal or approximately equal, then thedirection is direction 0. Direction 1 generally indicates that the pixelvalues are changing more in the horizontal direction than in thevertical direction, and direction 2 indicates that pixel values arechanging more in the vertical direction than in the horizontaldirection. Direction 0 indicates that the change in pixel values in thehorizontal direction is approximately equal to the change in pixelvalues in the vertical direction.

The determined direction metric (e.g. direction 0, direction 1,direction 2) can be used as a metric in the multi-metric filteringtechniques described in this disclosure. Using the example of FIG. 4Aagain, Metric 1 might be a variance metric, such as a sum-modifiedLaplacian value, while Metric 2 might be a direction determination asdescribed above. As described in reference to FIG. 4A, each of direction1, direction 2, and direction 0 can be associated with a range of Metric2 even though direction 1, direction 2, and direction 0 represent finitedeterminations instead of a spectrum of values.

In addition to using only horizontal activity and vertical activity asdescribed above, techniques of this disclosure also include using45-degree diagonal activity and 135-degree diagonal activity, asdescribed in equations 4 and 5, to determine directions, based on thefollowing conditions:

Direction=1, if 45deg_activity>k1*135deg_activity

Direction=2, if 135deg_activity>k2*45deg_activity

Direction=0, otherwise.

Direction determinations based on 45-degree diagonal activity and135-degree diagonal activity can be used as a metric with anothermetric, such as a sum-modified Laplacian value, as described above.

Additionally, a direction metric may also be determined, based on thefollowing conditions:

-   -   Direction=1, if 45deg_activity>k1*135deg_activity,        k2*Hor_activity, AND k3*Ver_activity    -   Direction=2, if 135deg_activity>>k4*45deg_activity,        k5*Hor_activity, AND k6*Ver_activity    -   Direction=3, if Hor_activity>k7*Ver_activity,        k8*135deg_activity, AND k9*45deg_activity    -   Direction=4, if Ver_activity>k10*Hor_activity,        k11*135deg_activity, AND k12*45deg_activity    -   Direction=0, otherwise.        As described above, k1 through k12 are constants selected to        determination how much greater than one of horizontal activity,        vertical activity, 45-degree activity, and 135-degree activity        needs to be compared to the others in order for a certain        direction to be selected. Direction determinations based on        horizontal activity, vertical activity, 45-degree diagonal        activity, and 135-degree diagonal activity can be used as a        metric with another metric, such as a sum-modified Laplacian        value, as described above.

Another metric that can be used with the techniques of this disclosureincludes an edge metric. An edge metric generally quantifies activitythat might be indicative of the presence of an edge in a block ofpixels. An edge may occur, for example, in a block of pixels if thatblock of pixels contains the boundary of an object within an image. Oneexample of edge detection includes using a current pixel's fourneighboring pixels (e.g., left, right, top, bottom) or using the currentpixel's eight neighboring pixels (left, right, top, bottom, top right,top left, bottom right, bottom left). Additionally, edge type detectionmay include using two neighboring pixels, such as top and bottom, leftand right, top left and bottom right, or top right and left bottom.

The pseudo code below shows examples of how edge information can becomputed for a current pixel (x, y) by comparing a pixel value (Rec),such as intensity, of the current pixel to the pixel values of thoseneighboring pixels (i.e., 4/8 pixels).

An EdgeType variable is initiated to 0. Each time a statement is true,the EdgeType variable is either incremented by 1 (as shown in the pseudocode by EdgeType ++) or decremented by 1 (as shown in the pseudo code byEdgeType −−). Rec[x][y] refers to a pixel value, such as the pixelintensity, of the pixel located at (x, y). The first grouping of “if”statements are for comparing the current pixel to top, bottom, left, andright neighbors. The second grouping of “if” statements are forcomparing the current pixel to the top-left, top-right, bottom-left, andbottom-right neighbors. The techniques of this disclosure can beimplemented using either group or both groups.

-   -   EdgeType=0;    -   if (Rec[x][y]>Rec[x−1][y]) EdgeType ++;    -   if (Rec[x][y]<Rec[x−1][y]) EdgeType −−;    -   if (Rec[x][y]>Rec[x+1][y]) EdgeType ++;    -   if (Rec[x][y]<Rec[x+1][y]) EdgeType −−;    -   if (Rec[x][y]>Rec[x][y−1]) EdgeType ++;    -   if (Rec[x][y]<Rec[x][y−1]) EdgeType −−;    -   if (Rec[x][y]>Rec[x][y+1]) EdgeType ++;    -   if (Rec[x][y]<Rec[x][y+1]) EdgeType −−;    -   if (Rec[x][y]>Rec[x−1][y−1]) EdgeType ++;    -   if (Rec[x][y]<Rec[x−1][y−1]) EdgeType −−;    -   if (Rec[x][y]>Rec[x+1][y−1]) EdgeType ++;    -   if (Rec[x][y]<Rec[x+1][y−1]) EdgeType −−;    -   if (Rec[x][y]>Rec[x−1][y+1]) EdgeType ++;    -   if (Rec[x][y]<Rec[x−1][y+1]) EdgeType −−;    -   if (Rec[x][y]>Rec[x+1][y+1]) EdgeType ++;    -   if (Rec[x][y]<Rec[x+1][y+1]) EdgeType −−;

If a current pixel is a local maximum, then the pixel value of the pixelwill be greater than all its neighbors and will have an edge type of 4if using four neighbors or an edge type of 8 if using eight neighbors.If a current pixel is local minimum, then the pixel value of the pixelwill be less than all its neighbors and will have an edge type of −4 ifusing four neighbors or an edge type of −8 if using eight neighbors.Thus, using the example techniques described above for determining anedge type between −4 and 4 or −8 and 8 can be used in determining afilter. The values determined for the edge type (i.e. values of −4 to 4or values of −8 to 8) can be mapped to ranges of a metric, such asMetric 1 or Metric 2 of FIG. 4A. In some implementations, absolutevalues of the edge type determination might be mapped to ranges, suchthat an edge type of −3 and 3, for example, would map to the samefilter.

The calculations of the various metrics described in this disclosure areonly intended to be examples and are not exhaustive. For example, themetrics can be determined using windows or lines of pixels that includemore neighboring pixels than described in this disclosure.

Additionally, in some implementations, the metrics described in thisdisclosure may be calculated using sub-sampling of the pixels in aparticular line or window. For example, to calculate a block activitymetric for a 4×4 block of pixels, metrics for activity and direction canbe calculated as follows:

-   -   Direction Metric        -   Ver_act(i,j)=abs (X(i,j)<<1−X(i,j−1)−X(i,j+1))        -   Hor_act(i,j)=abs (X(i,j)<<1−X(i−1,j)−X(i+1,j))        -   H_(B)=Σ_(i=0,2) Σ_(j=0,2) Hor_act(i,j)        -   V_(B)=Σ_(i=0,2) Σ_(j=0,2) Vert_act(i,j)        -   Direction=0, 1 (H_(B)>k1*V_(B)), 2 (V_(B)>k2*H_(B))    -   Activity Metric        -   L_(B)=H_(B)+V_(B)        -   5 classes (0, 1, 2, 3, 4)    -   Metric        -   Combination of Activity and Direction (e.g. 15 or 16            combinations as explained above in the example of FIG. 4B)

Hor_act (i, j) generally refers to the horizontal activity of currentpixel (i, j), and Vert_act(i, j) generally refers to the verticalactivity of current pixel (i,j). X(i, j) generally refers to a pixelvale of pixel (i, j). H_(B) refers to the horizontal activity of the 4×4block, which in this example is determined based on a sum of horizontalactivity for pixels (0, 0), (0, 2), (2, 0), and (2, 2). V_(B) refers tothe vertical activity of the 4×4 block, which in this example isdetermined based on a sum of vertical activity for pixels (0, 0), (0,2), (2, 0), and (2, 2). “<<1” represents a multiply by two operation. Asexplained above, based on the values of H_(B) and V_(B), a direction canbe determined. Using the example above, if the value of H_(B) is morethan k times the value of V_(B), then the direction can be determined tobe direction 1 (i.e. horizontal), which might correspond to morehorizontal activity than vertical activity. If the value of V_(B) ismore than k times the value of H_(B), then the direction can bedetermined to be direction 2 (i.e. vertical), which might correspond tomore vertical activity than horizontal activity. Otherwise, thedirection can be determined to be direction 0 (i.e. no direction),meaning neither horizontal nor vertical activity is dominant. The labelsfor the various directions and the ratios used to determine thedirections merely constitute one example, as other labels and ratios canalso be used.

Activity (L_(B)) for the 4×4 block can be determined as a sum of thehorizontal and vertical activity. The value of L_(B) can be classifiedinto a range, as described above. This particular example shows fiveranges although more or fewer ranges may similarly be used. Based on thecombination of activity and direction, a filter for the 4×4 block ofpixels can be selected. As described above, a filter may be selectedbased on a two-dimensional mapping of activity and direction to filters,as described in reference to FIGS. 4A and 4B, or activity and directionmay be combined into a single metric, and that single metric may be usedto select a filter.

FIG. 6A represents a 4×4 block of pixels. Using the sub-samplingtechniques described above, only four of the sixteen pixels are used.The four pixels are pixel (0, 0) which is labeled as pixel 601, pixel(2, 0) which is labeled as pixel 602, pixel (0, 2) which is labeled aspixel 603, and pixel (2, 2) which is labeled as pixel 604. TheHorizontal activity of pixel 601 (i.e. hor_act(0, 0)), for example, isdetermined based on a left neighboring pixel and a right neighboringpixel. The right neighboring pixel is labeled as pixel 605. The leftneighboring pixel is located in a different block than the 4×4 block andis not shown on FIG. 6A. The vertical activity of pixel 602 (i.e.ver_act(2, 0)), for example is determined based on an upper neighboringpixel and a lower neighboring pixel. The lower neighboring pixel islabeled as pixel 606, and the upper neighboring pixel is located in adifferent block than the 4×4 block and is not shown in FIG. 6A.

Generally using the same techniques described above, a block activitymetric may also be calculated using a different subset of pixels asfollows:

-   -   Direction Metric        -   Ver_act(i,j)=abs (X(i,j)<<1−X(i,j−1)−X(i,j+1))        -   Hor_act(i,j)=abs (X(i,j)<<1−X(i−1,j)−X(i+1,j))        -   H_(B)=Σ_(i=1,2) Σ_(j=1,2) H(i,j)        -   V_(B)=Σ_(i=1,2) Σ_(j=1,2) V(i,j)        -   Direction=0, 1(H>k1*V), 2 (V>k2*H)    -   Activity Metric        -   L_(B)=H_(B)+V_(B)        -   5 classes (0, 1, 2, 3, 4)    -   Metric        -   Combination of Activity and Direction (e.g. 15 or 16            combinations as explained above in the example of FIG. 4B)

This different subset of pixels for calculating H_(B) and V_(B) includespixels (1, 1), (2, 1), (1, 2), and (2, 2), shown on FIG. 6B as pixels611, 612, 613, and 614, respectively. As can be seen by FIG. 6B, all ofthe upper neighboring pixels, lower neighboring pixels, rightneighboring pixels, and left neighboring pixels for pixels 611, 612,613, and 614 are located within the 4×4 block. In the example of FIG.6B, pixels 611, 612, 613, and 614 are all located in the interior of theblock as opposed to be locating on the block boundary. Pixels 601, 602,603, and 605 in FIG. 6A and pixels 621, 624, 625, and 628 in FIG. 6C areexamples of pixels located on the block boundary. In otherimplementations, additional different subsets of pixel may be chosen.For example, subsets may be selected such that upper and lowerneighboring pixels for the pixels of the subset are within the 4×4block, but some left and right neighboring pixels are in neighboringblocks. Subsets may also be selected such that left and rightneighboring pixels for the pixels of the subset are within the 4×4block, but some upper and lower neighboring pixels are in neighboringblocks.

Generally using the same techniques described above, a block activitymetric may also be calculated using a subset of eight pixels as follows:

-   -   Direction Metric        -   Ver_act(i,j)=abs (X(i,j)<<1−X(i,j−1)−X(i,j+1))        -   Hor_act(i,j)=abs (X(i,j)<<1−X(i−1,j)−X(i+1,j))        -   H_(B)=Σ_(i=0, 1,2, 3) Σ_(j=1,2) H(i,j)        -   V_(B)=Σ_(i=0, 1, 2, 3) Σ_(j=1,2) V(i,j)        -   Direction=0, 1(H>k1*V), 2 (V>k2*H)    -   Activity Metric        -   L_(B)=H_(B)+V_(B)        -   5 classes (0, 1, 2, 3, 4)    -   Metric        -   Combination of Activity and Direction (e.g. 15 or 16            combinations as explained above in the example of FIG. 4B)

This different subset of eight pixels for calculating H_(B) and V_(B)includes pixels (0, 1), (1, 1), (2, 1), (3, 1), (0, 2), (1, 2), (2, 2),and (3, 2), shown on FIG. 6C as pixels 621, 622, 623, and 624, 625, 626,627, and 628 respectively. As can be seen by FIG. 6C, all of the upperneighboring pixels and lower neighboring pixels for pixels 621, 622,623, and 624, 625, 626, 627, and 628 are located within the 4×4 block,although pixels 621 and 625 each have left neighboring pixels in a leftneighboring block and pixels 624 and 628 each have right neighboringpixels in a right neighboring block. This particular selection of pixelsmay reduce encoder and/or decoder complexity by avoiding the need for aline buffer for storing pixel values of upper and/or lower neighboringblocks. Due to the left-to-right, top-to-bottom raster scan order, linebuffers for pixel values of upper and lower neighboring blocks oftenneed to store pixel values for the entire upper or lower line, which inthe case of the 1080P video, for example, might be 1920 pixels. Linebuffers for, left and right neighboring blocks, however, often only needto store pixel values for one LCU or a couple of LCUs, which might onlybe 64 or 128 pixels, for example. Thus, line buffers for pixel values ofupper and lower neighboring blocks may need to be significantly largerthan line buffers used for pixel values of left and right neighboringblocks. The selection of pixels shown in FIG. 6C may be able to avoidthe use of line buffers for pixel values of upper and lower neighboringblock, thus reducing coding complexity.

The examples of FIGS. 6A-6C are merely introduced techniques of thisdisclosure. It is contemplated that these techniques can be extended toblocks other than just 4×4 and that different subsets of pixels may beselected.

When computing a block activity metric, instead of original pixels,quantized pixels (i.e., X(i,j)>>N) can be used to reduce the complexityof operations, such as addition operations. Additionally, calculationscan be absolute difference based instead of Laplacian based. Forexample, when computing Hor_act(i,j) or Ver_act(i,j), absolutedifferences can be used instead of Laplacian values, as follows:

-   -   Direction Metric        -   Ver_act(i,j)=abs (X(i,j)−X(i,j−1))        -   Hor_act(i,j)=abs (X(i,j)−X(i−1,j))        -   H_(B)=Σ_(i=0,1,2) Σ_(i=0,1,2) H(i,j)        -   V_(B)=Σ_(i=0,1,2) Σ_(j=0,1,2) V(i,j)        -   Direction=0, 1(H>2V), 2 (V>2H)    -   Activity Metric        -   L_(B)=H_(B)+V_(B)        -   5 classes (0, 1, 2, 3, 4)    -   Metric        -   Activity+Direction (e.g. 15 or 16 combinations as explained            above in in the example of FIG. 4B)

This disclosure has described sub-sampling techniques with reference toa limited group of specific metrics. It is contemplated, however, thatthese sub-sampling techniques are generally applicable to other metrics,such as the other metrics discussed in this disclosure, that may be usedfor purposes of determining a filter. Additionally, although thesub-sampling techniques of this disclosure have been described withreference to 4×4 blocks of pixels, the techniques may also be applicableto blocks of other sizes.

FIG. 7 is a flow diagram illustrating a video coding techniqueconsistent with this disclosure. The techniques described in FIG. 7 canbe performed by the filter unit of a video encoder or a video decoder,such as filter unit 349 of video encoder 350 or filter unit 559 of videodecoder 560. The filter unit determines a first metric for a group ofpixels within a block of pixels (710). The first metric may, forexample, be an activity metric such as a sum-modified Laplacian value,or the first metric may be a direction metric. The first metric may bedetermined, for example, based on a comparison of the set of pixels inthe block, or based on a subset of the pixels in the block, to otherpixels in the block. The filter unit further determines a second metricfor the block (720). The second metric may, for example, be a directionmetric that is determined based on comparing a measure of horizontalactivity to a measure of vertical activity. Based on the first metricand the second metric, the filter unit determines a filter (730). Thefilter unit generates a filtered image by applying the filter to theblock (740). As discussed above, in some implementations, the block maybe a 2×2, 4×4, or M×N block of pixels, used for determining the firstmetric or the second metric. In some implementations, the first metricmay be a horizontal activity metric while the second metric is avertical activity metric, or the first metric may be an edge metricwhile the second metric is a direction metric.

FIG. 8A is a flow diagram illustrating video coding techniquesconsistent with this disclosure. The techniques described in FIG. 8A canbe performed by the filter unit of a video decoder, such as filter unit559 of video decoder 560. Filter unit 559 maps a first range combinationto a first filter (810A). The first range combination is combination ofa first range of values for a first metric and a first range of valuesfor a second metric. The first metric may, for example, be asum-modified Laplacian value and the second metric may be a directionmetric, although others metrics may also be used. Filter unit 559 maps asecond range combination to a second filter (820A). The second rangecombination is a combination of a second range of values for the firstmetric and a second range of values for the second metric. Filter unit559 then maps a current range combination to a filter based on areceived codeword. The current range combination includes the firstrange of values of the first metric and the second range of values forthe second metric. If the codeword is a first codeword (830A, yes), thenfilter unit 559 maps the current range combination to the first filter(840A). The first codeword indicates the current range combination ismapped to the same filter as the first range combination. If thecodeword is a second codeword (850A, yes), the filter unit 559 maps thecurrent range combination to the second filter (860A). The secondcodeword indicates the current range combination is mapped to the samefilter as the second combination. If the codeword is neither a firstcodeword nor a second codeword (850A, no), then filter unit 559 maps thecurrent range combination to a third filter (870A). If in response toreceiving a third codeword, wherein the third codeword identifies thatthird filter. In the example of FIG. 8A, the first codeword and thesecond codeword may each include fewer bits than the third codeword.

FIG. 8B is a flow diagram illustrating video coding techniquesconsistent with this disclosure. The techniques described in FIG. 8B canbe performed by the filter unit of a video decoder, such as filter unit559 of video decoder 560. Filter unit 559 generates a mapping of rangecombinations to filters (810B). Each range combination, for example, caninclude a range for a first metric and a range for a second metric. Inresponse to receiving a first codeword that signals a current rangecombination is mapped to a same filter as a previous range combination(820B, yes), filter unit 559 maps the current range combination to thesame filter as the previous range combination (830B). In response toreceiving a second codeword that signals the current range combinationis mapped to a different filter than the previous range combination(820B, no), filter unit 559 maps the current range combination to a newfilter (840B). As described above, the current range combination can bedetermined based on a known transmission order. In some examples, thenew filter can be identified based on the second codeword, while inother examples, the new filter might be determined based on the order inwhich filter coefficients are signaled.

FIG. 9A is a flow diagram illustrating video coding techniquesconsistent with this disclosure. The techniques described in FIG. 9A canbe performed by the filter unit of a video encoder, such as filter unit349 of video encoder 350. Filter unit 349 determines a mapping of rangecombinations to filters (910A). Each range combination includes a rangeof values for a first metric and a range of values for a second metric.For a current range combination, if a current range combination ismapped to the same filter as a previous range combination that comprisesthe same range of values for the first metric (920A, yes), then filterunit 349 generates a first codeword (930A). If the current rangecombination is mapped to the same filter as a previous range combinationthat comprises the same range of values for the second metric (940A,yes), then filter unit 349 generates a second codeword (950A). If thecurrent range combination is not mapped to either the previous rangecombination that comprises the same range of values for the first metricor the previous range combination that comprises the same range ofvalues for the second metric (950A, no), then filter unit 349 generatesa third codeword (960A). The third codeword can identify a filter mappedto the current range combination.

FIG. 9B is a flow diagram illustrating video coding techniquesconsistent with this disclosure. The techniques described in FIG. 9BAcan be performed by the filter unit of a video encoder, such as filterunit 349 of video encoder 350. Filter unit 349 determines a mapping ofrange combinations to filters (910B). Each range combination can, forexample, include a range for a first metric and a range for a secondmetric. When a current range combination being coded has the same filteras a previously coded range combination (920B, yes), filter unit 349 cangenerate a first codeword to signal that the current range combinationis mapped to the same filter as a previous range combination (930B).When a current range combination being coded does not have the samefilter as a previously coded range combination (920B, no), filter unit349 can generating a second codeword (940B). The second codeword canidentify the filter mapped to the current range combination. Asdescribed above, the current range combination can be determined basedon a known transmission order. In the example of FIG. 9B, the firstcodeword may include fewer bits than the second codeword.

In the examples of FIGS. 8A and 8B and FIGS. 9A and 9B, the terms “firstcodeword,” “second codeword,” and “third codeword” are used todifferentiate between different codewords and not meant to imply asequential ordering of codewords.

FIG. 10 is a flow diagram illustrating video coding techniquesconsistent with this disclosure. The techniques described in FIG. 10 canbe performed by the filter unit of a video encoder, such as filter unit349 of video encoder 350, or the filter unit of a video decoder, such asfilter unit 559. The filter unit determines a mapping of rangecombinations to filters (1010). The range combinations include a rangefor a first metric and a range for a second metric. The filter unitdetermines a unique range combination identification (ID) for each rangecombination (1020). The unique range combination IDs correspond tosequential values. The filter unit assigns a first unique group ID to afirst group of range combinations based on the sequential value of arange combination ID of at least one range combination in the firstgroup of range combinations (1030). The groups of range combinationsinclude range combinations mapped to the same filter, the unique groupIDs correspond to a set of sequential values. The filter unit codes afirst set of filter coefficients corresponding to the same filter basedon the sequential value of the first unique filter ID (1040). In thecase of video encoder, coding the first set of filter coefficients caninclude, for example, signaling the filter coefficients in an encodedbitstream using differential coding techniques. In the case of a videodecoder, coding the first set of filter coefficients can includereconstructing the filter coefficients based on information received inan encoded bitstream.

FIG. 11 is a flow diagram illustrating video coding techniquesconsistent with this disclosure. The techniques described in FIG. 11 canbe performed by the filter unit of a video encoder, such as filter unit349 of video encoder 350, or the filter unit of a video decoder, such asfilter unit 559. The filter unit determines a mapping of rangecombinations to filters (1110). The range combinations can include arange for a first metric and a range for a second metric. Each rangecombination can have a unique range combination identification (ID), andeach unique range combination ID can correspond to a sequential valuefor the range combination. The filter unit can assigns a unique group IDto each group of range combinations (1120). The filter unit can assignthe unique group IDS, for example, based on the sequential values of therange combinations. A group of range combinations can includes rangecombinations mapped to a common filter, and the unique group IDs cancorrespond to a set of sequential values. The filter unit can code setsof filter coefficients for the filters based on the unique group IDs(1140).

In the example of FIG. 11, the filter unit can assign the unique groupIDs by, for example, assigning a unique group ID corresponding to alowest sequential value of the unique group IDs to a group of rangecombinations that comprises a range combination with a range combinationID corresponding to a lowest sequential value of the range combinationIDs. In another example, the filter unit can assign the unique group IDcorresponding to a highest sequential value of the unique group IDs to agroup of range combinations that comprises a range combination with arange combination ID corresponding to a highest sequential value of therange combination IDs.

In instances where the filter unit is part of a video decoder, thefilter unit can code the sets of filter coefficients by generating thesets of filter coefficients based on information received in a codedbitstream. The filter unit can, for example, generate the sets of filtercoefficients using differential coding techniques. In instances wherethe filter unit is part of a video encoder, the filter unit can code thesets of filter coefficients by signaling the sets of filter coefficientsin a coded bitstream in an order selected based on the sequential valuesof the unique group IDs. The filter unit can, for example, signal thesets of filter coefficients using differential coding techniques.

The foregoing disclosure has been simplified to some extent in order toconvey details. For example, the disclosure generally describes sets offilters being signaled on a per-frame or per-slice basis, but sets offilters may also be signaled on a per-sequence basis, per-group ofpicture basis, per-group of slices basis, per-CU basis, per-LCU basis,or other such basis. In general, filters may be signaled for anygrouping of one or more CUs. Additionally, in implementation, there maybe numerous filters per input per CU, numerous coefficients per filter,and numerous different levels of variance with each of the filters beingdefined for a different range of variance. For example, in some casesthere may be sixteen or more filters defined for each input of a CU andsixteen different ranges of variance corresponding to each filter.Additionally, when this disclosure describes transmitting filterinformation, it should not be assumed that all filter information istransmitted at the same coding level. For example, in someimplementations, some filter information such as filter descriptionsyntax may be signaled on a frame-by-frame basis or slice-by-slice basiswhile other filter information such as filter coefficients are signaledon an LCU-by-LCU basis. Syntax at other levels of the coding hierarchy,such as sequence level, GOP-level, or other levels could also be definedfor conveying some or all of such filter information

Each of the filters for each input may include many coefficients. In oneexample, the filters comprise two-dimensional filters with 81 differentcoefficients defined for a filter support that extends intwo-dimensions. However, the number of filter coefficients that aresignaled for each filter may be fewer than 81 in some cases. Coefficientsymmetry, for example, may be imposed such that filter coefficients inone dimension or quadrant may correspond to inverted or symmetric valuesrelative to coefficients in other dimensions or quadrants. Coefficientsymmetry may allow for 81 different coefficients to be represented byfewer coefficients, in which case the encoder and decoder may assumethat inverted or mirrored values of coefficients define othercoefficients. For example, the coefficients (5, −2, 10, 10, −2, 5) maybe encoded and signaled as the subset of coefficients (5, −2, 10). Inthis case, the decoder may know that these three coefficients define thelarger symmetric set of coefficients (5, −2, 10, 10, −2, 5).

The techniques of this disclosure may be implemented in a wide varietyof devices or apparatuses, including a wireless handset, and integratedcircuit (IC) or a set of ICs (i.e., a chip set). Any components, modulesor units have been described provided to emphasize functional aspectsand does not necessarily require realization by different hardwareunits.

Accordingly, the techniques described herein may be implemented inhardware, software, firmware, or any combination thereof. If implementedin hardware, any features described as modules, units or components maybe implemented together in an integrated logic device or separately asdiscrete but interoperable logic devices. If implemented in software,the techniques may be realized at least in part by a computer-readablemedium comprising instructions that, when executed in a processor,performs one or more of the methods described above. Thecomputer-readable medium may comprise a computer-readable storage mediumand may form part of a computer program product, which may includepackaging materials. The computer-readable storage medium may compriserandom access memory (RAM) such as synchronous dynamic random accessmemory (SDRAM), read-only memory (ROM), non-volatile random accessmemory (NVRAM), electrically erasable programmable read-only memory(EEPROM), FLASH memory, magnetic or optical data storage media, and thelike. The techniques additionally, or alternatively, may be realized atleast in part by a computer-readable communication medium that carriesor communicates code in the form of instructions or data structures andthat can be accessed, read, and/or executed by a computer.

The code may be executed by one or more processors, such as one or moredigital signal processors (DSPs), general purpose microprocessors, anapplication specific integrated circuits (ASICs), field programmablelogic arrays (FPGAs), or other equivalent integrated or discrete logiccircuitry. Accordingly, the term “processor,” as used herein may referto any of the foregoing structure or any other structure suitable forimplementation of the techniques described herein. In addition, in someaspects, the functionality described herein may be provided withindedicated software modules or hardware modules configured for encodingand decoding, or incorporated in a combined video codec. Also, thetechniques could be fully implemented in one or more circuits or logicelements.

Various aspects of the disclosure have been described. These and otheraspects are within the scope of the following claims.

1. A method of video coding, the method comprising: determining a firstmetric for a block of pixels, wherein the first metric is determinedbased on a comparison of a subset of the pixels in the block to otherpixels in the block; based on the first metric, determining a filter forthe block of pixels; and generating a filtered image by applying thefilter to the block of pixels.
 2. The method of claim 1, whereindetermining the first metric comprises comparing a measure of horizontalactivity to a measure of vertical activity.
 3. The method of claim 2,wherein the horizontal activity is determined based on comparing a pixelvalue of at least one pixel in the subset to a pixel value of a leftneighboring pixel and a pixel value of a right neighboring pixel,wherein the left neighboring pixel and the right neighboring pixel areboth in the block of pixels.
 4. The method of claim 2, wherein thevertical activity is determined based on comparing a pixel value of atleast one pixel in the subset to a pixel value of an upper neighboringpixel and a pixel value of a lower neighboring pixel, wherein the upperneighboring pixel and the lower neighboring pixel are both in the blockof pixels.
 5. The method of claim 1, wherein the block of pixels is anM×N block of pixels, and the subset of pixels comprises pixels not onthe boundary of the M×N block.
 6. The method of claim 1, wherein theblock of pixels is a 4×4 block of pixels including sixteen pixels, andwherein the subset of pixels includes four pixels.
 7. The method ofclaim 1, further comprising determining a second metric for the block ofpixels; and based on the first metric and the second metric, determininga filter for the block of pixels.
 8. The method of claim 1, wherein thesecond metric comprises an activity metric.
 9. The method of claim 1,wherein the method of video coding implements a Quadtree-based AdaptiveLoop Filter (QALF) scheme with multiple filters.
 10. The method of claim1, wherein the method is performed by a video coding device comprising avideo encoder.
 11. The method of claim 1, wherein the method isperformed by a video coding device comprising a video decoder.
 12. Avideo coding device comprising: a filter unit configured to: determine afirst metric for a block of pixels, wherein the first metric isdetermined based on a comparison of a subset of the pixels in the blockto other pixels in the block; determine a filter for the block of pixelsbased on the first metric; and generate a filtered image by applying thefilter to the block of pixels a memory configured to store a filteredresult of the filter unit; and a memory configured to store a filteredresult of the filter unit.
 13. The video coding device of claim 12,wherein determining the first metric comprises comparing a measure ofhorizontal activity to a measure of vertical activity.
 14. The videocoding device of claim 13, wherein the horizontal activity is determinedbased on comparing a pixel value of at least one pixel in the subset toa pixel value of a left neighboring pixel and a pixel value of a rightneighboring pixel, and wherein the left neighboring pixel and the rightneighboring pixel are both in the block of pixels.
 15. The video codingdevice of claim 13, wherein the vertical activity is determined based oncomparing a pixel value of at least one pixel in the subset to a pixelvalue of an upper neighboring pixel and a pixel value of a lowerneighboring pixel, and wherein the upper neighboring pixel and the lowerneighboring pixel are both in the block of pixels.
 16. The video codingdevice of claim 12, wherein the block of pixels is an M×N block ofpixels, and the subset of pixels comprises pixels not on the boundary ofthe M×N block.
 17. The video coding device of claim 12, wherein theblock of pixels is a 4×4 block of pixels including sixteen pixels, andwherein the subset of pixels includes four pixels.
 18. The video codingdevice of claim 12, wherein the filter unit is further configured to:determine a second metric for the block of pixels; and determine afilter for the block of pixels based on the first metric and the secondmetric.
 19. The video coding device of claim 18, wherein the secondmetric comprises an activity metric.
 20. The video coding device ofclaim 12, wherein the filter unit implements a Quadtree-based AdaptiveLoop Filter (QALF) scheme with multiple filters.
 21. The video codingdevice of claim 12, wherein the video coding device comprises a videoencoder.
 22. The video coding device of claim 12, wherein the videocoding device comprises a video decoder.
 23. An apparatus comprising:means for determining a first metric for a block of pixels, wherein thefirst metric is determined based on a comparison of a subset of thepixels in the block to other pixels in the block; means for determininga filter for the block of pixels based on the first metric; and meansfor generating a filtered image by applying the filter to the block ofpixels.
 24. The apparatus of claim 23, wherein determining the firstmetric comprises comparing a measure of horizontal activity to a measureof vertical activity.
 25. The apparatus of claim 24, wherein thehorizontal activity is determined based on comparing a pixel value of atleast one pixel in the subset to a pixel value of a left neighboringpixel and a pixel value of a right neighboring pixel, and wherein theleft neighboring pixel and the right neighboring pixel are both in theblock of pixels.
 26. The apparatus of claim 24, wherein the verticalactivity is determined based on comparing a pixel value of at least onepixel in the subset to a pixel value of an upper neighboring pixel and apixel value of a lower neighboring pixel, and wherein the upperneighboring pixel and the lower neighboring pixel are both in the blockof pixels.
 27. The apparatus of claim 23, wherein the block of pixels isan M×N block of pixels, and the subset of pixels comprises pixels not onthe boundary of the M×N block.
 28. The apparatus of claim 23, whereinthe block of pixels is a 4×4 block of pixels including sixteen pixels,and wherein the subset of pixels includes four pixels.
 29. The apparatusof claim 23, further comprising means for determining a second metricfor the block of pixels; and means for determining a filter for theblock of pixels based on the first metric and the second metric.
 30. Acomputer-readable storage medium having stored thereon instructions thatwhen executed cause one or more processors to: determine a first metricfor a block of pixels, wherein the first metric is determined based on acomparison of a subset of the pixels in the block to other pixels in theblock; determine a filter for the block of pixels based on the firstmetric; and generate a filtered image by applying the filter to theblock of pixels.